OS and Software > All Things Unix > FreeBSD 9 - ZFS - iSCSI

reply to scrummie02

Re: FreeBSD 9 - ZFS - iSCSI

Thanks. I can manually partition so I'll take that into account. I already have it installed with a disk pool running. I'm running tests right now and plan to do the same with openindiana today or tomorrow. I'll post the results of my tests.

I'm aware the iscsi implementation ported from NetBSD isn't that great. So I'm using the istgt port and it seems to be okay.

I'm running tests with hdparm and dd along with running multiple installs of CentOS and Windows to see my performance hits.

I haven't trolled the lists that long, I'm usually on the OpenBSD lists and don't really pay attention to the FreeBSD ones as much.

PS. The ZFS versions on OpenIndiana/Illumos are the same as FreeBSD. So I'm aware of only one thing not being available on FreeBSD and that is the ability to export the ZFS volume to iSCSI directly from ZFS. Not that it makes a difference, Comstar seems to the "go to" pkg for iSCSI on the opensolaris based distributions.
--
"He's a politician. It's like being a hooker. You can't be one unless you can pretend to like people while you're f*cking them."
General Welfare

I understand. The ZFS versions -- as in the metadata version and on-disk formats -- are the same, but the code and how ZFS actually behaves on FreeBSD is not the same. This comes up on the freebsd-fs list quite often when people have issues, who then go and install IllumOS using the same disks/pool and the issue is gone. There are other known idiocies in FreeBSD's ZFS implementation, such as how dedup and compression are handled and how they destroy system interactivity, which are not a problem on Solaris, OpenSolaris, OpenIndiana, or IllumOS (and the root cause/reason is known -- again, the FreeBSD code is not the same).

I look forward to reading about your encounters with iSCSI and ZFS on FreeBSD, but as I said, check the mailing lists for the past 3-4 months. Topics come up there quite often about problems with both technologies, and in some cases problems when both are used together.
--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer.

Okay, as for performance they're about the same.

With a 1gig NIC and a zfs mirror with 2x SATA2 drives I'm getting an average of about 105-110 MB/s write and about the same for read.

This is using dd on both read and write operations. I have three VM's operating on the LUN, two CentOS and one Windows 2008R2. With Windows update running on the WIndows server, software updates to the CentOS desktop and running dd on the CentOS server I'm achieving a lot of the same performance stats.

I will post more detail on that later.

I was expecting a little better performance with Comstar and Openindiana (OI) though, kind of disappointed really. The only issue's I've run into is I lost my iSCSI connection from the ESX host to the OI host a couple of times that caused corruption in the guest OSes. This kind of concerned me. I also noted RAM usage was higher for OI than FreeBSD.

To be fair, at first I left the defaults for ZFS in place for OI and had to tune a little bit such as setting the arc min and arc max. Here is the ZFS variables I changed on both installs:

vfs.zfs.prefetch_disable=0

vfs.zfs.arc_min="1024M"

vfs.zfs.arc_max="3584M"

On OI the settings similar and are set in /etc/system/

I'm going to leave the FreeBSD running overnight because with the OI install I got checksum errors on my zpool today after testing last night and this morning.
--
"He's a politician. It's like being a hooker. You can't be one unless you can pretend to like people while you're f*cking them."
General Welfare

reply to koitsu
Oh, forgot to mentioned, LZJB compression enabled on both ZFS pools.