一、qemu中物理内存的注册

cpu_register_physical_memory 

 -->cpu_notify_set_memory 

 ---->kvm_client_set_memory 

 ------>kvm_set_phys_mem 

 -------->kvm_set_user_memory_region 

 ---------->kvm_vm_ioctl(进入内核)


内核中会调用kvm_vm_ioctl_set_memory_region最终调用到__kvm_set_memory_region函数
在 kvm_set_memory_region函数中有如下代码:

slots->memslots[mem->slot] = new; 

   old_memslots = kvm->memslots; 

   rcu_assign_pointer(kvm->memslots, slots); 

   synchronize_srcu_expedited(&kvm->srcu);


因此函数 kvm_set_memory_region本质是创建并填充了一个临时kvm_memslots结构,并把其赋值给kvm->memslots(全局的)。

二、处理用户态虚拟的地址(主要考虑tlb不能命中的情况)
1、查物理tlb如果不能命中会调用host中do_kvm_tlbmiss
2、do_kvm_tlbmiss会先判断地址是IO地址还是访存的地址,如果是访存地址,会进一步查guest tlb表,如果查guest tlb还没有命中,就会把guest tlb miss异常注入到guest系统中,guest kernel会根据页表来填充guest tlb,当guest调用TLBWI特权指令时,会再次陷入host中,调用do_kvm_cpu异常处理
3、在do_kvm_cpu中模拟TLBWI指令,先填充guest tlb 表项,在调用kvmmips_update_shadow_tlb来更新物理tlb(shadow tlb)
4、在kvmmips_update_shadow_tlb中,通过gfn_to_page和page_to_phys两个函数将gpa转化成hpa,再将hpa填充到物理tlb中
5、gfn_to_page函数(我想讲的重点)这个函数会调用到gfn_to_hva
6、gfn_to_hva调用gfn_to_memslot和gfn_to_hva_memslot
gfn_to_memslot代码如下:

struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn) { 

   int i; 

   struct kvm_memslots *slots = kvm_memslots(kvm); 

   for (i = 0; i < slots->nmemslots; ++i) { 

     struct kvm_memory_slot *memslot = &slots->memslots[i]; 

     if (gfn >= memslot->base_gfn 

      && gfn < memslot->base_gfn + memslot->npages) 

       return memslot; 

   } 

   return NULL; 

 }


代码中首先调用kvm_memslots获得slots,kvm_memslots代码如下:

static inline struct kvm_memslots *kvm_memslots(struct kvm *kvm) { 

   return rcu_dereference_check(kvm->memslots, 

     srcu_read_lock_held(&kvm->srcu) 

     || lockdep_is_held(&kvm->slots_lock)); 

 } 

 本质是return kvm->memsolts。 

 gfn_to_hva_memslot代码如下: 

 static unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot, gfn_t gfn) { 

  return slot->userspa

ce_addr + (gfn - slot->base_gfn) * PAGE_SIZE;
}
由此看来gpa到hva的关键是slot->userspace_addr,其在qemu中kvm_set_user_memory_region中通过qemu_safe_ram_ptr函数赋值。

qemu_safe_ram_ptr代码如下: 

 void *qemu_safe_ram_ptr(ram_addr_t addr) { 

   RAMBlock *block; 

   QLIST_FOREACH(block, &ram_list.blocks, next) { 

     if (addr - block->offset < block->length) { 

       return block->host + (addr - block->offset); 

     } 

   } 

   fprintf(stderr, "Bad ram offset %" PRIx64 "\n", (uint64_t)addr); 

   abort(); 

   return NULL; 

 }


因此得找到block->host,在qemu的qemu_ram_alloc_from_ptr函数中赋值,在该函数中有这么一句话new_block->host = qemu_vmalloc(size);从host系统中分配一个hva地址。

结论:
综合以上分析可以看出,在qemu中调用qemu_ram_alloc主要是分配RAMBlock结构,并将其插入ram_list.blocks链表,它的本质上分配了一个hva地址,把它放到RAMBlock结构host域;调用cpu_register_physical_memory主要填充struct kvm结构的slots域,它的本质是将一个gha地址与hva地址对应起来,将hva放在slot->userspace_addr中,将gha放在slot->base_gfn中。quma通过上面两个函数就把一段gha的空间映射成一段hva空间。

三、console显示过程(基于cirrusfb)
先看一个函数栈:

[<4000000080451164>] cirrusfb_imageblit+0xa0/0x284 

 [<400000008043ce5c>] bit_putcs+0x3dc/0x48c 

 [<400000008046eb8c>] do_update_region+0x148/0x1a4 

 [<40000000804705f4>] update_region+0xb4/0xdc 

 [<40000000804393bc>] fbcon_switch+0x5b8/0x61c 

 [<4000000080470ef4>] redraw_screen+0x188/0x2a8 

 [<4000000080472c84>] take_over_console+0x368/0x3cc 

 [<4000000080436030>] fbcon_takeover+0x108/0x188 

 [<4000000080160204>] notifier_call_chain.isra.1+0x40/0x90 

 [<4000000080160540>] __blocking_notifier_call_chain+0x48/0x68 

 [<400000008042ee8c>] register_framebuffer+0x2b0/0x2dc 

 [<400000008010f4b4>] cirrusfb_pci_register+0x608/0x6c4 

 [<400000008042650c>] pci_device_probe+0x60/0xa0 

 [<4000000080489008>] driver_probe_device+0x108/0x1f0 

 [<400000008048915c>] __driver_attach+0x6c/0xa4 

 [<40000000804879f8>] bus_for_each_dev+0x54/0xa0 

 [<40000000804881ec>] bus_add_driver+0xf0/0x310 

 [<4000000080489838>] driver_register+0xe0/0x194 

 [<4000000080426214>] __pci_register_driver+0x5c/0x11c 

 [<4000000080886710>] cirrusfb_init+0x164/0x198 

 [<4000000080870c18>] do_one_initcall+0xbc/0x204 

 [<4000000080870ecc>] kernel_init+0x16c/0x244 

 [<40000000801189e8>] kernel_thread_helper+0x10/0x18


从函数栈可以看出,register_framebuffer会触发一个FB_EVENT_FB_REGISTERED事件,调用函数fbcon_fb_registered,该函数中调用fbcon_takeover来接管操作console的函数,从此之后console的操作,就会调用下面函数

static const struct consw fb_con = { 

   .owner             = THIS_MODULE, 

   .con_startup      = fbcon_startup, 

   .con_init          = fbcon_init, 

   .con_deinit       = fbcon_deinit, 

   .con_clear        = fbcon_clear, 

   .con_putc         = fbcon_putc, 

   .con_putcs        = fbcon_putcs, 

   .con_cursor       = fbcon_cursor, 

   .con_scroll       = fbcon_scroll, 

   .con_bmove        = fbcon_bmove, 

   .con_switch       = fbcon_switch, 

   .con_blank        = fbcon_blank, 

   .con_font_set     = fbcon_set_font, 

   .con_font_get     = fbcon_get_font, 

   .con_font_default = fbcon_set_def_font, 

   .con_font_copy    = fbcon_copy_font, 

   .con_set_palette  = fbcon_set_palette, 

   .con_scrolldelta  = fbcon_scrolldelta, 

   .con_set_origin   = fbcon_set_origin, 

   .con_invert_region = fbcon_invert_region, 

   .con_screen_pos   = fbcon_screen_pos, 

   .con_getxy        = fbcon_getxy, 

   .con_resize       = fbcon_resize, 

   .con_debug_enter  = fbcon_debug_enter, 

   .con_debug_leave  = fbcon_debug_leave, 

 };


我们不防以fbcon_putcs函数为例,进一步分析,其代码如下:

static void fbcon_putcs(struct vc_data *vc, const unsigned short *s,int count, int ypos, int xpos) { 

   struct fb_info *info = registered_fb[con2fb_map[vc->vc_num]]; 

   struct display *p = &fb_display[vc->vc_num]; 

   struct fbcon_ops *ops = info->fbcon_par; 

   if (!fbcon_is_inactive(vc, info)) 

     ops->putcs(vc, info, s, count, real_y(p, ypos), xpos, 

      get_color(vc, info, scr_readw(s), 1), 

      get_color(vc, info, scr_readw(s), 0)); 

 }


它需要调用info->fbcon_par->putcs(info 的数据结构是struct fb_info),info->fbcon_par初始化在函数是fbcon_set_bitops,函数如下:

void fbcon_set_bitops(struct fbcon_ops *ops){ 

   ops->bmove = bit_bmove; 

   ops->clear = bit_clear; 

   ops->putcs = bit_putcs; 

   ops->clear_margins = bit_clear_margins; 

   ops->cursor = bit_cursor; 

   ops->update_start = bit_update_start; 

   ops->rotate_font = NULL; 

   if (ops->rotate) 

   fbcon_set_rotate(ops); 

 }


因此会继续调用bit_putcs,其最终会调用到info->fbops->fb_imageblit(info, image);(info 的数据结构是struct fb_info),info->fbops的初始化函数是cirrusfb_set_fbinfo中,该函数中有info

->fbops = &cirrusfb_ops;一句话,cirrusfb_ops结构如下: 

 static struct fb_ops cirrusfb_ops = { 

   .owner         = THIS_MODULE, 

   .fb_open       = cirrusfb_open, 

   .fb_release    = cirrusfb_release, 

   .fb_setcolreg  = cirrusfb_setcolreg, 

   .fb_check_var  = cirrusfb_check_var, 

   .fb_set_par    = cirrusfb_set_par, 

   .fb_pan_display = cirrusfb_pan_display, 

   .fb_blank      = cirrusfb_blank, 

   .fb_fillrect   = cirrusfb_fillrect, 

   .fb_copyarea   = cirrusfb_copyarea, 

   .fb_sync       = cirrusfb_sync, 

   .fb_imageblit  = cirrusfb_imageblit, 

 };


因此最终会调用到cirrusfb_imageblit函数。

上面一个过程就是一个console写操作,最终调到cirrusfb驱动中cirrusfb_imageblit过程。



四、xserver 显示(基于fbmem)



xserver 下普通的显卡驱动,通常会直接操作寄存器,具体操作就是,先mmap(/dev/mem)io空间的基址,再通过基址加偏移的方式,操作寄存器。


但fbmem是个例外,其不用操作既存器,而是通过ioctl(/dev/fb0),把这些操作丢给内核去做。


但是两者在都没有加速的情况下,framebuffer操作方式是相同的,都是将framebuffer区域通过mmap映射到用户态,然后交给xorg中其他代码处理映射后地址。



五、结论



qemu中cirrus_linear_writeb函数在console下调用很多次而在Xserver不被调用原因如下:



首先在xserver下,我们将framebuffer区域(0x14000000开始的一段区域)mmap成了guest虚拟地址(通过调用/dev/fb0 mmap函数),也就说在xserver所有frambuffer的操作,都是通过这个gva.


其次qema中,在函数map_linear_vram中,通过cpu_register_physical_memory(s->vga.map_addr, s->vga.map_end - s->vga.map_addr, s->vga.vram_offset);(将0x14000000这个gpa和一个hva建立起了联系)


因此在xserver下整个framebuffer操作全部成了内存操作,不是IO操作,过程是gva->gpa->hva->hpa,不会回到qemu中,当然也就不可能访问qemu中的cirrus_linear_writeb函数了。


再次在console下,访console操作最终会调用到cirrusfb_imageblit函数,在cirrusfb_imageblit中有这么一句话memcpy(info->screen_base, image->data, size);其中info->screen_base就是0x14000000remap后(IO空间),因此会回到qemu中,调用cirrus_linear_writeb。


最后,为什么这么关注cirrus_linear_writeb函数,因为在qemu中操作framebuffer表现在往s->vga.vram_ptr中写或从s->vga.vram_ptr读,(s->vga.vram_ptr就是我们说的hva),通过2242 s->vram_offset = qemu_ram_alloc(NULL, "vga.vram", vga_ram_size);



2243 s->vram_ptr = qemu_get_ram_ptr(s->vram_offset);得到。只有在cirrus_linear_writeb函数中,才在往s->vga.vram_ptr这个hva写后,通过cpu_physical_memory_set_dirty将这个区域标记,而在我们更新屏幕是,dirty的区域是我们更新的判断条件。