Creating and mounting a fileshare as a file in a KVM Windows guest

Yeah, the title is a mouthfull.

So, let's get it done.

1. Create the disk, which is henceforth called foretest.iso. I made it to be 1G because, as the name says, it is a test.

   raub@vmhost:~$ dd if=/dev/zero of=foretest.iso bs=1G count=1
   1+0 records in
   1+0 records out
   1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.29044 s, 469 MB/s
   raub@vmhost:~$

   NOTE: Some people would complain that I am using the .iso extension for my image file instead of, say, .img. Well, this is my article; deal with it.

2. Now let's feed it to the vm.

   raub@vmhost:~$ virsh attach-disk desktop dev/foretest.iso hda --type raw --mode readwrite                 
   error: No support for readwrite in command 'attach-disk'
   
   raub@vmhost:~$

   Well, I did not know my KVM host does not have attach-disk. I guess I need to use attach-device instead. But, we will need to use a config file to describe how we want the storage to look like. If it looks familiar, we have used attach-disk to mount a USB device, namely an APC UPS.

   So, here is the .xml file:

   cat > dev/foretest.xml << 'EOF'
   <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/home/raub/dev/foretest.iso'/>
   <target dev='hdc'/>
   </disk >
   EOF

   And here we are feeding it into the guest:

   raub@vmhost:~$ sudo virsh attach-device --config testdesktop dev/foretest.xml
   Device attached successfully
   
   raub@vmhost:~$

   So we should be able to go to the guest and see the drive waving at us, right? Er, not quite. it is being listed here

   <disk type='file' device='disk'>
   
         <source file='/home/raub/dev/foretest.iso'/>
         <target dev='vdd' bus='virtio'/>
         <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
      </disk>

   But not in the GUI thingie nor inside the guest. And, yes, I have 2 virtual CD drives in this guest.

   
   

On a second thought, maybe I am doing this wrong. You see, the --config option tells it to add the drive to the config file. Only way to enable that is to completely shut down the vm guest (rebooting is not enough) and then restart it. What if I use --live instead, which does the deed in real time, as if I just pop the computer on and add the drive to an unused sata port?

raub@vmhost:~$ sudo virsh attach-device --live testdesktop dev/foretest.xml
error: Failed to attach device from dev/foretest.xml
error: internal error unable to execute QEMU command '__com.redhat_drive_add': Device 'drive-virtio-disk3' could not be initialized

raub@vmhost:~$

I guess it does not like me to add hard drives to a running guest. We need to try something else.

Attempt #2: USB

What if we pretend our disk is really a removable device like a USB drive? We begin by rewriting our .xml file:

cat > dev/foretest.xml << 'EOF'
<disk type='file' device='disk'>
  <driver name='qemu' type='raw'/>
  <source file='/home/raub/dev/foretest.iso'/>
  <target dev='sdd' bus='usb'/>
</disk>
EOF

Now let's mount it:

raub@desktop:~$ sudo virsh attach-device --live testdesktop dev/foretest.xml
Device attached successfully

raub@desktop:~$

So far so good. Can our guest see it this time? When I use Windows Explorer (do notice I disabled one of the CD drives as I was doing something else between the last screen capture and this one) I do not see the drive there; I kinda expected Windows to say something like "Hey! You just connected an unformatted drive! I need to format it!" Oh well; there are more than one way to get this done:

There it is! And we can format it!

And then, we can put things into it.

Now, let's unmount it

raub@desktop:~$ sudo virsh detach-device --live testdesktop dev/foretest.xml
Device detached successfully

raub@desktop:~$

Now let's make it a physical drive!

Stupid question: since we pretend our foretest.iso file is a USB drive, can we make it a real USB drive? We would test that question by first grabbing a USB drive which is at least 1GB in size (I had a 16GB one doing nothing), which was mounted into my KVM host as /dev/sdh.And now we need to copy the file to the drive.

raub@desktop:~$ sudo dd if=dev/foretest.iso of=/dev/sdh
raub@desktop:~$

Then grab USB drive and then mount it in a windows physical desktop. And it will look just like how it looks in the testdesktop guest down to be able to write to it.

The next step will be to see if we can make it a system disk, a disk we can boot a computer from. But, that will be for another article.

Finding disk space hogs in a Windows server/workstation

If you have any doubts, we will be doing it from the command line. Just want to put that out before we start. Also, it turned out this is a long and boring article; deal with it.

So, where were we? Disk space and what is using it. That is a problem common to all OS: you have a partition without infinite disk space (sorry ZFS, it happens sometimes) and is running out of space:

• Sometimes it is a careless user; some OS allow you to tell non-system programs, like the ones run by a user, can only use up to 95% of the disk. This way we have some space to fix things.
  
• Sometimes it is actually a program being run as a system/root/admin account, which is trouble since it can use up the entire partition.

If the machine in question is a server, a Windows server since that is what we wrote in the name of the article, we might not be able to just ignore it; others will be affected by this. So, how to take care of this problem? The lazy fix is to throw more space at it and move on. The proper solution is to find out who is hoarding all the space and why. I would like to talk about doing the right thing.

The standard Windows approach would be to search for some app online, which must have a graphics interface and ideally from some site with a name like "finddiskusage.com" because such domain names do inspire confidence, right? Specially when the site's text is pretty much "You do not know what is using your disk space? Click here to download the solution!" Any relationship with a phishing email is merely coincidental.

So, after downloading this shady program from the suspicious website, we then install it and make sure to turn off the firewall and run it with admin rights. And after it does what we hope it is supposed to do, we then take a screenshot of the output and paste it to our documentation.

I do not know about you but I really do not like to install programs in any server, be it windows/linux/mac/solaris/aix/whatever. I think they should have only the bare minimum to do their job; you should see my Linux servers. Since I am the one writing this article, I will put my dictator hat and look for something that fits my style.

In Unix in general and Linux as a special case there is a program called du which allows you to check the disk usage at a given location. You can be short and only show, say, how much all the files and directories (folders in Windows) inside a given directory, or go recursively and show detailed views for every single directory inside the original one. Output is text, which means you can feed it to something else like sort or some program that will make a decision based on the data.

It would be really cool if there was something like that in Windows. One can dream...

Thing is, we do not have to wait for unicorns and fairies to come up with a solution. Nor we have to reinvent the wheel. You see,

1. There is something like that natively for windows. I do like cygwin but that requires installing yet another collection of packages that need to be patched and upgraded. Kinda wasteful if all you want is little du. I believe less is more.
   
2. It is called du just like the unix one.
   
3. You do not need to look for it in some shady or just compromised site. You can find it right at Microsoft as part of the sysinternals package(s).
   
4. You do not even need to install anything. Just put its directory somewhere you want to use it, including a USB or network drive, and run it from there.

Not bad if you ask me. Enough talk, let's use it.

Using du

The most common way I use du in Windows is like I use in Linux:

C:\Documents and Settings\raub>"\Documents and Settings\raub\My Documents\DU\du.exe" -l 1 \windows >> du.log

Du v1.5 - report directory disk usage
Copyright (C) 2005-2013 Mark Russinovich
Sysinternals - www.sysinternals.com


C:\Documents and Settings\raub>

Ok, it is old but it is not like it is getting more and more useless features to make it bloated. Like interfacing with your bluetooth-enabled IoT-based massage chair. Let me show it in action with a real (!) example: at work I have a Windows 10 vm for desktop. And as you can see, it ran out of disk disk space:

That's not much free space left! If you know Windows, it will get really slow when its boot/OS disk is that full. I am going to assume I should first check the Users directory. If I am wrong, I would then check the Windows one, First I would like to make a point I will be running du.exe off a network fileshare, strongbadia

PS C:\Users\raub> ls \\spacemoose\users\raub\bin


    Directory: \\strongbadia\users\raub\bin


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        5/21/2018   9:32 AM         169072 du.exe
-a----        5/21/2018   9:32 AM         191616 du64.exe
-a----        6/28/2018   4:06 PM            543 GetDirSize4DateRange.ps1
-a----        7/18/2017  10:12 AM         854072 putty.exe


PS C:\Users\raub>

You are now getting to learn a few secret things about me! Yes I have a bin dir. Inside it you can see the du.exe and du64.exe. Both are very tiny compared to the crazy GUI programs you can get off suspicious sites to do the very same thing. And, that is all you need: those two files. Well, I will be running du.exe even though my guest is a 64bit windows vm. Because I can. So, let's see what is in the root dir for the users dir:

PS C:\Users\raub> \\strongbadia\users\raub\bin\du.exe -l 1 c:\users\

DU v1.61 - Directory disk usage reporter
Copyright (C) 2005-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

      31,480  c:\users\bob.adm
       2,665  c:\users\Default
  51,390,386  c:\users\raub
     629,717  c:\users\raub.adm
       2,727  c:\users\raub.tst
       2,768  c:\users\Public
     129,592  c:\users\windows-user
Files:        2429983
Directories:  9581
Size:         54,003,166,071 bytes
Size on disk: 59,917,407,544 bytes

PS C:\Users\raub>

Man! My homedir is full of junk. What is that and where is it? Let's check it:

PS C:\Users\raub> \\strongbadia\users\raub\bin\du.exe -l 1 c:\users\raub\

DU v1.61 - Directory disk usage reporter
Copyright (C) 2005-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

  46,023,020  c:\users\raub\AppData
           0  c:\users\raub\Contacts
           1  c:\users\raub\Desktop
           3  c:\users\raub\dev
   1,836,295  c:\users\raub\Documents
   3,551,535  c:\users\raub\Downloads
           0  c:\users\raub\eqlgroupmgr
           0  c:\users\raub\Favorites
           1  c:\users\raub\Links
           0  c:\users\raub\Music
           0  c:\users\raub\OneDrive
       5,012  c:\users\raub\Pictures
           0  c:\users\raub\Saved Games
           3  c:\users\raub\Searches
           0  c:\users\raub\Videos
Files:        2425559
Directories:  3976
Size:         52,663,166,132 bytes
Size on disk: 58,521,224,472 bytes

PS C:\Users\raub>

AppData! A conveniently normally invisible source of many out of space drives. Not to du it is. Let's keep going in:

PS C:\Users\raub> \\strongbadia\users\raub\bin\du.exe -l 1 c:\users\raub\appdata\

DU v1.61 - Directory disk usage reporter
Copyright (C) 2005-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

  45,780,916  c:\users\raub\appdata\Local
       2,588  c:\users\raub\appdata\LocalLow
     198,212  c:\users\raub\appdata\Roaming
Files:        2424970
Directories:  3898
Size:         47,085,278,621 bytes
Size on disk: 52,935,700,880 bytes

PS C:\Users\raub> \\strongbadia\users\raub\bin\du.exe -l 1 c:\users\raub\appdata\local\
[...]
PS C:\Users\raub> \\strongbadia\users\raub\bin\du.exe -l 1 C:\users\raub\appdata\local\Microsoft\Windows\INetCac
he\\

DU v1.61 - Directory disk usage reporter
Copyright (C) 2005-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

         239  c:\users\raub\appdata\local\microsoft\windows\inetcache\Content.MSO
         878  c:\users\raub\appdata\local\microsoft\windows\inetcache\Content.Outlook
       5,207  c:\users\raub\appdata\local\microsoft\windows\inetcache\Content.Word
      18,852  c:\users\raub\appdata\local\microsoft\windows\inetcache\IE
  39,825,165  c:\users\raub\appdata\local\microsoft\windows\inetcache\Low
           0  c:\users\raub\appdata\local\microsoft\windows\inetcache\Virtualized
           0  c:\users\raub\appdata\local\microsoft\windows\inetcache\WebTempDir
Files:        2410022
Directories:  55
Size:         40,806,752,248 bytes
Size on disk: 46,635,979,024 bytes

PS C:\Users\raub> \\strongbadia\users\raub\bin\du.exe -l 1 C:\users\raub\appdata\local\Microsoft\Windows\INetCache\low\

DU v1.61 - Directory disk usage reporter
Copyright (C) 2005-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

  39,819,877  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\IE
Files:        2408915
Directories:  34
Size:         40,780,969,849 bytes
Size on disk: 46,606,749,696 bytes

PS C:\Users\raub>

That smells like that other nemesis of web browsers: Internet Explorer or Edge. What is inside that dir?

PS C:\Users\raub> ls C:\users\raub\appdata\local\Microsoft\Windows\INetCache\Low\IE\
PS C:\Users\raub> dir C:\users\raub\appdata\local\Microsoft\Windows\INetCache\Low\IE\
PS C:\Users\raub>

WTF? Why can't I see what is inside it? Let me du inside it:

PS C:\Users\raub> \\strongbadia\users\raub\bin\du.exe -l 1 C:\users\raub\appdata\local\Microsoft\Windows\INetCache\low\

DU v1.61 - Directory disk usage reporter
Copyright (C) 2005-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

   1,237,378  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\ie\10EAXOSV
   1,246,204  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\ie\145KIAQM
   1,235,782  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\ie\2F8FLUJD
   1,241,300  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\ie\2VB3GL0K
   1,250,215  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\ie\5BZQD406
   1,244,360  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\ie\5IVFOAV9
[...]
   1,251,347  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\ie\TQ26RPSG
   1,236,991  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\ie\VQH18R7G
   1,240,357  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\ie\VYHW6URF
   1,239,508  c:\users\raub\appdata\local\microsoft\windows\inetcache\low\ie\XYKKE45S
Files:        2408912
Directories:  33
Size:         40,775,554,697 bytes
Size on disk: 46,601,322,496 bytes

PS C:\Users\raub>

Before you ask, I am using a powershell window, where ls and dir behave the same. I came from unix so you can understand which one I prefer. So we have a ton (33) of stupid cache folders that the browser could not be bothered to delete after it quit. Thanks, Microsoft, for not cleaning after itself. Really. And, everything below c:\users\raub\appdata\local\microsoft\windows\inetcache\low is hidden (?). Alright then, off it goes! Command line cares not about hidden paths! Note: get-help can be seen as the powershell equivalent of the Unix/linux man.

PS C:\Users\raub> get-help rm

NAME
    Remove-Item

SYNOPSIS
    Deletes files and folders.


SYNTAX
    Remove-Item [-Confirm] [-Credential ] [-Exclude ] [-Filter ] [-Force] [-Include
    ] -LiteralPath  [-Recurse] [-Stream ] [-UseTransaction] [-WhatIf]
    []

    Remove-Item [-Path]  [-Confirm] [-Credential ] [-Exclude ] [-Filter ]
    [-Force] [-Include ] [-Recurse] [-Stream ] [-UseTransaction] [-WhatIf] []
   
    Remove-Item [-Stream ] []
   

DESCRIPTION
    The Remove-Item cmdlet deletes one or more items. Because it is supported by many providers, it can delete many
    different types of items, including files, folders, registry keys, variables, aliases, and functions.
    In file system drives, the Remove-Item cmdlet deletes files and folders.
   
    If you use the Stream dynamic parameter, it deletes the specified alternate data stream, but does not delete the
    file.
   
    Note: This custom cmdlet help file explains how the Remove-Item cmdlet works in a file system drive. For
    information about the Remove-Item cmdlet in all drives, type "Get-Help Remove-Item -Path $null" or see Remove-Item
    at http://go.microsoft.com/fwlink/?LinkID=113373.
   

RELATED LINKS 
    Online version: http://technet.microsoft.com/library/jj628241(v=wps.630).aspx
    Remove-Item (generic); http://go.microsoft.com/fwlink/?LinkID=113373
    FileSystem Provider
    Clear-Content
    Get-Content
    Get-ChildItem
    Get-Content
    Get-Item
    Remove-Item
    Set-Content
    Test-Path


REMARKS
    To see the examples, type: "get-help Remove-Item -examples".
    For more information, type: "get-help Remove-Item -detailed".
    For technical information, type: "get-help Remove-Item -full".
    For online help, type: "get-help Remove-Item -online"


PS C:\Users\raub> rm -force -recurse C:\users\raub\appdata\local\Microsoft\Windows\INetCache\low\IE

Note that since we are running this from the command line, I did not have to do the usual screenshot Windows blog and articles love so much. I could cut and paste the real output and put it here. I could also have piped it into some other script to use the output for its nefarious uses. FYI, the above command has been running for 38 minutes now and has not finished yet.

So, what I have done above in this real example can be also used with servers since

1. We are using Microsoft published program.
   
2. The program is self-contained and requires no installation.
   
3. The program fears no hidden directories.
   
4. The program has very small footprint.
   
5. The program can be run locally, off a USB, or from a network drive.

If you can put up with not having a cute window with some animation, I think sysinternals' version of du is a nice tiny add-on to a Windows server manager arsenal.

Thoughts on creating a large iso using dd

Let's see if I can avoid being long winded as usual:

So I wanted to create a .iso file that was going to be treated like a disk (more of that part in a future thread). Nothing better to do the deed but that old friend, dd:

raub@desktop:/export/scratch$ dd if=/dev/zero of=olddrive.iso bs=44G count=1
dd: memory exhausted by input buffer of size 47244640256 bytes (44 GiB)
raub@desktop:/export/scratch$

It so happens that dd is trying to create a 44G file by first allocating that in the RAM. And that will not work for me as I only have 32G total of memory.

The solution is rather obvious if you stop to think about it (which, I must admit, I did not): just don't use all the RAM! Instead, use smaller chunks and repeat that a lot. How about 1G chunks 44 times?

raub@desktop:/export/scratch$ dd if=/dev/zero of=olddrive.iso bs=1G count=44
44+0 records in
44+0 records out
47244640256 bytes (47 GB, 44 GiB) copied, 354.018 s, 133 MB/s
raub@desktop:/export/scratch$

Much better!

Resizing a VirtualBox .vdi file from command line

Another quick one: Let's say I am running VirtualBox (or vbox for short) and have a Microsoft Windows virtual client. Let's date ourselves and make it a Windows 7 vm. And I just found out its disk, win7.vdi is a bit small (40GB)and I would like to up it a bit, say, to 60GB. And me being the klutz I am, or have this unexplained desire to do everything on command line so I can script it later, I do not know how to do that using the GUI. So, like many times before, cue in for some command line action!

The command we will be using today is vboxmanage modifymedium or, as it used to be called, vboxmanage modifyhd. Let's see which options we have avaiable:

Oracle VM VirtualBox Command Line Management Interface Version 5.0.16
(C) 2005-2016 Oracle Corporation
All rights reserved.

Usage:

VBoxManage modifymedium     [disk|dvd|floppy] 
                            [--type normal|writethrough|immutable|shareable|
                                    readonly|multiattach]
                            [--autoreset on|off]
                            [--property ]
                            [--compact]
                            [--resize |--resizebyte ]

Why we would modify a dvd? Well this is really not a real dvd, but one of those .vdi files, which you can build to pretend to be a DVD. Other formats are also supported but .vdi is the native one. Now, I usually try to run my examples in a Linux host unless they specifically only run on a different OS (try running Powershell in Linux). Since VirtualBox can run in OSX, Windows, and Linux, I chose Windows this time. So, let's try to add 20GB to it:

PS C:\Program Files\Oracle\VirtualBox> .\vboxmanage modifyhd \
'C:\users\raub\VirtualBox VMs\win7\win7.vdi' --resize +20480
0%...
Progress state: VBOX_E_NOT_SUPPORTED
VBoxManage.exe: error: Resize medium operation for this format is not implemented yet!
PS C:\Program Files\Oracle\VirtualBox>

You probably guessed I do not have the virtualbox binary path added to my default path, so I cheated and went to that directory. Of course you could have provided the full path to the binary, which would not be as lazy. In any case, it did not like my pretending this is a lv and I was expanding it of 20GB (i.e. the "+" did not work out). So, let's just tell it to do 60GB (I am cheating with the disk size) total and be done.

PS C:\Program Files\Oracle\VirtualBox> .\vboxmanage modifyhd \
'C:\users\raub\VirtualBox VMs\win7\win7.vdi' --resize 60000
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
PS C:\Program Files\Oracle\VirtualBox>

Of course now we have to resize the drive within the vm client itself, and that depends on the OS that is being run by that client.

notes:

• I merged all snapshots before doing this. I did not test whether this would work with "snapshoted" disks.
  
• I have read before about people having issues with vmdk files. I do not know if that still happens or not, but I thought it was worth mentioning. If you have errors, just do as the first reply and then convert it to vdi do your resizing, and then convert it back to vdmk.
  
• I like to give the full path to the .vdi fle.
  

Adding a disk to a libvirt/kvm vm client

So you have a kvm virtualization infrastructure which you, being lazy like me, manage using libvirt. You then created a vm client with the virtual disk partitioned just right (using LVM or not; pick your poison). But later you realized you needed another disk. Maybe it is because you need to have data encrypted to meet HIPAA or PCI requirements. Maybe you just want to keep your data in a different drive. The point is you need another drive and don't want to/should not resize the current virtual disk associated with the VM.
So, first thing you do is create the new disk. If you are using KVM, chances are you have been using qcow2 disks. Or maybe vmdk, iSCSI, or, like me, lvm. No matter what, you probably know how to create a new disk, so I will assumed you took time to figure out how large it needs to be to fit your needs and created the little bastard. Because, as I mentioned before, I am lazy, I will say we are running libvirt in linux, with a vm client called vmclient and creating LVs to use as virtual disks. So, we need a 10GB virtual disk that we'll call data because we are friends with Captain Obvious.

lvcreate -L 10G -n data vmhost_vg0

creates as we know a 10GB lv as /dev/vmhost_vg0/data. To make it easier on us, we will shut down the vm client. Once that is done (do check it using virsh list --all, will you?), we then run

virsh edit vmclient

Note we could have created a properly configured xml file and fed into the config, but I do forget how to make it properly configured so I prefer to cheat.
When the config file is open, look for the disk entries; they should look like this:

    <disk device="disk" type="file">
      <driver cache="none" io="native" name="qemu" type="raw"/>
      <source file="/dev/vmhost_vg0/vmclient_boot"/>
      <target bus="virtio" dev="vda"/>
      <address bus="0x00" domain="0x0000" function="0x0" slot="0x04" type="pci"/>
    </disk>

which is how our original virtual disk is configured to be used in this vm client. Of course, if you were using qcow, vmdk, or something else, the entry might look a bit different; make a note of how it differs from my example and move on. This is why I said I like to cheat: I can see how the old disk was defined and copy that instead of trying to figure out how to do it.

Now you need to add the new drive. As you guessed from the above, we will do it by copying the above entry and changing it a bit. Now, we do not need to copy everything; we just need enough so virsh knows what we want. It will fill the blanks. So, after we copy the relevant bits below the already defined disk and change the drive name, we would have something like this

    <disk device="disk" type="file">
      <driver cache="none" io="native" name="qemu" type="raw"/>
      <source file="/dev/vmhost_vg0/data"/>
      <target bus="virtio" dev="vda"/>
    </disk>

Save it; it should close without issues. If not, got back and see if you missed something. If it did not bark, use virsh dump xml vmclient to see your handiwork. Mine looks like this:

    <disk device="disk" type="file">
      <driver cache="none" io="native" name="qemu" type="raw"/>
      <source file="/dev/vmhost_vg0/vmclient_boot"/>
      <target bus="virtio" dev="vda"/>
      <address bus="0x00" domain="0x0000" function="0x0" slot="0x04" type="pci"/>
    </disk>
    <disk device="disk" type="file">
      <driver cache="none" io="native" name="qemu" type="raw"/>
      <source file="/dev/vmhost_vg0/data"/>
      <target bus="virtio" dev="vda"/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/%gt
    </disk>

As you can see, it added the pci bus address entry on its own. Now all we need to do is reboot the vmclient, format the new disk on whatever way we want, and start using it.

ZFSing around (or, using ZFS under Solaris)

ZFS is a different file system than the old UFS. Best way to describe it is to think on it as a file system with LVM built on it. If you know LVM, you will understand what I am trying to say and recognize a lot of things here. If not, I promise that soon I will try to talk about it and compare ZFS, Linux LVM, and AIX LVM. For now, accept my simplistic explanation:

Logical Volume Manager, or LVM, is a way the separate the physical storage (think hard drives and networks drives and such things) from the storage seen by the user. In other words, what the users see is a pseudo disk (or logical volume if we are to use its terminology) of sorts which can be made of a collection of other disks, raid arrays, or just partitions of those disks. Now, while Linux LVM provides the logical disk which is then formatted and partitioned as they were normal disks, in ZFS these two steps are done become just one.

I think this will make sense using an example: in this Solaris 10 machine, we have two 73GB SCA hard drives. One of which, c0t0d0 (yes, Sun likes to call their drives differently than everybody else. We will talk about that in some other episode. For now, it suffices to say that d0 is the SCSI disk with SCSI ID 0), was formated as ZFS during the installation of the operating system and currently looks like this:

# df -h
Filesystem             size   used  avail capacity  Mounted on
boot/ROOT/root          67G   6.8G    40G    15%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   1.9G   1.4M   1.9G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
sharefs                  0K     0K     0K     0%    /etc/dfs/sharetab
fd                       0K     0K     0K     0%    /dev/fd
boot/ROOT/root/var      67G   727M    40G     2%    /var
swap                   1.9G    32K   1.9G     1%    /tmp
swap                   1.9G    40K   1.9G     1%    /var/run
boot                    67G   176K    40G     1%    /boot
boot/export             67G    20K    40G     1%    /export
boot/export/home        67G    15G    40G    28%    /export/home
# 

Then we have c0t1d0, also a 73GB HD that we plan to add to the hard drive. For this machine, I would like to have the following layout:

/
/var
/tmp
/export/home   (accounts)
/export/hosts  (virtual hosts, in case we have them)
/export/images (as this may end up being a netboot/jumpstart server)

Each of those would reside in a separate partition. Some of that has already been taken care of by Solaris' default install, but we are left with /export/hosts and /export, which will be run off c0t1d0. Before ZFS, we would divide the hard drive using format into two partitions and would edit /etc/vfstab to mount the two partitions at boot time:

# cat /etc/vfstab                      
#device         device          mount           FS      fsck    mount   mount
#to mount       to fsck         point           type    pass    at boot options
#
fd      -       /dev/fd fd      -       no      -
/proc   -       /proc   proc    -       no      -
/dev/zvol/dsk/boot/swap -       -       swap    -       no      -
/devices        -       /devices        devfs   -       no      -
sharefs -       /etc/dfs/sharetab       sharefs -       no      -
ctfs    -       /system/contract        ctfs    -       no      -
objfs   -       /system/object  objfs   -       no      -
swap    -       /tmp    tmpfs   -       yes     -
/dev/dsk/c0t1d0s4     /dev/rdsk/c0t1d0s4      /export/hosts        ufs     2       yes     -
/dev/dsk/c0t1d0s6     /dev/rdsk/c0t1d0s6      /export/images       ufs     2       yes     -
# 

but, in ZFS things are a bit different. First we need to create the pseudo disk mentioned above. In ZFS terminology that is called a pool; I guess they want to remind us that you usually create it by adding disks to it. So we create the pool, which shall be named storagepool. By now you may have realized that boot is the name of the pool created during the installation of the operating system.

# zpool create  storagepool c0t1d0
# df -h
Filesystem             size   used  avail capacity  Mounted on
boot/ROOT/root          67G   6.8G    40G    15%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   1.9G   1.4M   1.9G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
sharefs                  0K     0K     0K     0%    /etc/dfs/sharetab
fd                       0K     0K     0K     0%    /dev/fd
boot/ROOT/root/var      67G   727M    40G     2%    /var
swap                   1.9G    32K   1.9G     1%    /tmp
swap                   1.9G    40K   1.9G     1%    /var/run
boot                    67G   176K    40G     1%    /boot
boot/export             67G    20K    40G     1%    /export
boot/export/home        67G    15G    40G    28%    /export/home
storagepool             67G    19K    67G     1%    /storagepool
#

You may have noticed that the pool we created, storagepool is also a filesystem: as soon as we created the pool, it becomes a filesystem mounted under /storagepool. That is different than what is done under Linux LVM, where you would first create the pool, then partition and format that pool into the partitions you are going to use, and then mount them using /etc/fstab (the Linux equivalent to /etc/vfstab and mount -a. In Solaris and ZFS, on the other hand, that is done with just one command. I mean, we did not even have to edit /etc/vfstab. The current list of ZFS mountpoints defined can be seen using zfs filelist:

# zfs list
NAME                 USED  AVAIL  REFER  MOUNTPOINT
boot                26.5G  40.5G   176K  /boot
boot/ROOT           7.46G  40.5G    18K  legacy
boot/ROOT/root      7.46G  40.5G  6.75G  /
boot/ROOT/root/var   727M  40.5G   727M  /var
boot/dump           2.00G  40.5G  2.00G  -
boot/export         15.0G  40.5G    20K  /export
boot/export/home    15.0G  40.5G  15.0G  /export/home
boot/swap              2G  42.5G    16K  -
storagepool         89.5K  66.9G     1K  /storagepool
# 

If you remember, from storagepool we will be creating /export/hosts. That is done as follows:

# zfs create storagepool/hosts
# df -h
Filesystem             size   used  avail capacity  Mounted on
boot/ROOT/root          67G   6.8G    40G    15%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   1.9G   1.4M   1.9G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
sharefs                  0K     0K     0K     0%    /etc/dfs/sharetab
fd                       0K     0K     0K     0%    /dev/fd
boot/ROOT/root/var      67G   727M    40G     2%    /var
swap                   1.9G    32K   1.9G     1%    /tmp
swap                   1.9G    40K   1.9G     1%    /var/run
boot                    67G   176K    40G     1%    /boot
boot/export             67G    20K    40G     1%    /export
boot/export/home        67G    15G    40G    28%    /export/home
storagepool             67G    19K    67G     1%    /storagepool
storagepool/hosts       67G    18K    67G     1%    /storagepool/hosts
#

If we just leave at that, a mount point, /storagepool/hosts, is created. But, we really do not want that. We want to mount storagepool/hosts in /export/hosts, and mount we shall.

# zfs set mountpoint=/export/hosts storagepool/hosts
# 

Now, it seems to be mounted where we want

# df -h
Filesystem             size   used  avail capacity  Mounted on
boot/ROOT/root          67G   6.8G    40G    15%    /
/devices                 0K     0K     0K     0%    /devices
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   1.9G   1.4M   1.9G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
sharefs                  0K     0K     0K     0%    /etc/dfs/sharetab
fd                       0K     0K     0K     0%    /dev/fd
boot/ROOT/root/var      67G   727M    40G     2%    /var
swap                   1.9G    32K   1.9G     1%    /tmp
swap                   1.9G    40K   1.9G     1%    /var/run
boot                    67G   176K    40G     1%    /boot
boot/export             67G    20K    40G     1%    /export
boot/export/home        67G    15G    40G    28%    /export/home
storagepool             67G    19K    67G     1%    /storagepool
storagepool/hosts       67G    18K    67G     1%    /export/hosts
#

Neat, huh? There is more we can do, like establish quotas and such stuff. But that will be for another episode.

Some parting thoughts

Remember that Sun call boot and storagepool pools. Even though we've only used one disk per pool, we could have used more. For instance, we could have grouped them together like in Linux LVM,

# zpool create  anotherpool c0t0d0 c0t1d0
# 

or, we could have made a pool where one of the disks mirrors the other, as in a raid 1

# zpool create  anotherpool mirror c0t0d0 c0t1d0
# 

Another option is instead of doing a mirror is to create a raid 5 (Sun claims it is a variation of raid 5, bigger, faster, better) using the raidz option and at least 3 drives (min for such raid),

# zpool create  anotherpool raidz c0t0d0 c0t1d0 c0t2d0
# 

We could also create hotspares for those arrays; check the man page for zpool for more info on that. I hope that will make you think on what ZFS can do and why I think it is rather neat.