kernel_optimize_test/fs/xfs
Dave Chinner ad1858d777 xfs: rework remote attr CRCs
Note: this changes the on-disk remote attribute format. I assert
that this is OK to do as CRCs are marked experimental and the first
kernel it is included in has not yet reached release yet. Further,
the userspace utilities are still evolving and so anyone using this
stuff right now is a developer or tester using volatile filesystems
for testing this feature. Hence changing the format right now to
save longer term pain is the right thing to do.

The fundamental change is to move from a header per extent in the
attribute to a header per filesytem block in the attribute. This
means there are more header blocks and the parsing of the attribute
data is slightly more complex, but it has the advantage that we
always know the size of the attribute on disk based on the length of
the data it contains.

This is where the header-per-extent method has problems. We don't
know the size of the attribute on disk without first knowing how
many extents are used to hold it. And we can't tell from a
mapping lookup, either, because remote attributes can be allocated
contiguously with other attribute blocks and so there is no obvious
way of determining the actual size of the atribute on disk short of
walking and mapping buffers.

The problem with this approach is that if we map a buffer
incorrectly (e.g. we make the last buffer for the attribute data too
long), we then get buffer cache lookup failure when we map it
correctly. i.e. we get a size mismatch on lookup. This is not
necessarily fatal, but it's a cache coherency problem that can lead
to returning the wrong data to userspace or writing the wrong data
to disk. And debug kernels will assert fail if this occurs.

I found lots of niggly little problems trying to fix this issue on a
4k block size filesystem, finally getting it to pass with lots of
fixes. The thing is, 1024 byte filesystems still failed, and it was
getting really complex handling all the corner cases that were
showing up. And there were clearly more that I hadn't found yet.

It is complex, fragile code, and if we don't fix it now, it will be
complex, fragile code forever more.

Hence the simple fix is to add a header to each filesystem block.
This gives us the same relationship between the attribute data
length and the number of blocks on disk as we have without CRCs -
it's a linear mapping and doesn't require us to guess anything. It
is simple to implement, too - the remote block count calculated at
lookup time can be used by the remote attribute set/get/remove code
without modification for both CRC and non-CRC filesystems. The world
becomes sane again.

Because the copy-in and copy-out now need to iterate over each
filesystem block, I moved them into helper functions so we separate
the block mapping and buffer manupulations from the attribute data
and CRC header manipulations. The code becomes much clearer as a
result, and it is a lot easier to understand and debug. It also
appears to be much more robust - once it worked on 4k block size
filesystems, it has worked without failure on 1k block size
filesystems, too.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Ben Myers <bpm@sgi.com>
Signed-off-by: Ben Myers <bpm@sgi.com>
2013-05-23 18:04:06 -05:00
..
Kconfig xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
kmem.c
kmem.h
Makefile xfs: split remote attribute code out 2013-04-27 12:49:32 -05:00
mrlock.h xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
time.h
uuid.c
uuid.h
xfs_acl.c
xfs_acl.h
xfs_ag.h xfs: add CRC checks to the AGI 2013-04-21 14:57:43 -05:00
xfs_alloc_btree.c xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_alloc_btree.h xfs: add support for large btree blocks 2013-04-21 14:53:46 -05:00
xfs_alloc.c xfs: Avoid pathological backwards allocation 2013-05-20 13:09:11 -05:00
xfs_alloc.h
xfs_aops.c xfs: fix sub-page blocksize data integrity writes 2013-05-20 14:14:25 -05:00
xfs_aops.h
xfs_attr_leaf.c xfs: rework remote attr CRCs 2013-05-23 18:04:06 -05:00
xfs_attr_leaf.h xfs: add CRCs to attr leaf blocks 2013-04-27 12:45:01 -05:00
xfs_attr_remote.c xfs: rework remote attr CRCs 2013-05-23 18:04:06 -05:00
xfs_attr_remote.h xfs: rework remote attr CRCs 2013-05-23 18:04:06 -05:00
xfs_attr_sf.h
xfs_attr.c xfs: split remote attribute code out 2013-04-27 12:49:32 -05:00
xfs_attr.h xfs: split remote attribute code out 2013-04-27 12:49:32 -05:00
xfs_bit.c
xfs_bit.h
xfs_bmap_btree.c xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_bmap_btree.h xfs: add support for large btree blocks 2013-04-21 14:53:46 -05:00
xfs_bmap.c xfs: buffer type overruns blf_flags field 2013-04-27 13:01:58 -05:00
xfs_bmap.h
xfs_btree.c xfs: buffer type overruns blf_flags field 2013-04-27 13:01:58 -05:00
xfs_btree.h xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_buf_item.c
xfs_buf_item.h xfs: buffer type overruns blf_flags field 2013-04-27 13:01:58 -05:00
xfs_buf.c xfs: rework remote attr CRCs 2013-05-23 18:04:06 -05:00
xfs_buf.h
xfs_cksum.h
xfs_da_btree.c xfs: xfs_da3_node_read_verify() doesn't handle XFS_ATTR3_LEAF_MAGIC 2013-05-20 16:32:30 -05:00
xfs_da_btree.h xfs: add buffer types to directory and attribute buffers 2013-04-27 13:01:06 -05:00
xfs_dfrag.c
xfs_dfrag.h
xfs_dinode.h xfs: add version 3 inode format with CRCs 2013-04-21 15:03:33 -05:00
xfs_dir2_block.c xfs: buffer type overruns blf_flags field 2013-04-27 13:01:58 -05:00
xfs_dir2_data.c xfs: buffer type overruns blf_flags field 2013-04-27 13:01:58 -05:00
xfs_dir2_format.h xfs: shortform directory offsets change for dir3 format 2013-04-27 12:24:32 -05:00
xfs_dir2_leaf.c xfs: fix missing KM_NOFS tags to keep lockdep happy 2013-05-20 16:18:05 -05:00
xfs_dir2_node.c xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_dir2_priv.h xfs: add buffer types to directory and attribute buffers 2013-04-27 13:01:06 -05:00
xfs_dir2_sf.c xfs: shortform directory offsets change for dir3 format 2013-04-27 12:24:32 -05:00
xfs_dir2.c
xfs_dir2.h
xfs_discard.c
xfs_discard.h
xfs_dquot_item.c
xfs_dquot_item.h
xfs_dquot.c xfs: add CRC checks for quota blocks 2013-04-21 14:58:22 -05:00
xfs_dquot.h xfs: xfs_dquot prealloc throttling watermarks and low free space 2013-03-22 16:06:30 -05:00
xfs_error.c xfs: increase hexdump output in xfs_corruption_error 2013-04-21 14:48:41 -05:00
xfs_error.h
xfs_export.c
xfs_export.h
xfs_extent_busy.c
xfs_extent_busy.h
xfs_extfree_item.c xfs: Don't reference the EFI after it is freed 2013-05-20 14:29:34 -05:00
xfs_extfree_item.h xfs: don't free EFIs before the EFDs are committed 2013-04-05 13:25:35 -05:00
xfs_file.c aio: don't include aio.h in sched.h 2013-05-07 20:16:25 -07:00
xfs_filestream.c
xfs_filestream.h
xfs_fs.h
xfs_fsops.c xfs: add CRC checks to the AGI 2013-04-21 14:57:43 -05:00
xfs_fsops.h
xfs_globals.c
xfs_ialloc_btree.c xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_ialloc_btree.h xfs: add support for large btree blocks 2013-04-21 14:53:46 -05:00
xfs_ialloc.c xfs: buffer type overruns blf_flags field 2013-04-27 13:01:58 -05:00
xfs_ialloc.h
xfs_icache.c
xfs_icache.h
xfs_inode_item.c xfs: add version 3 inode format with CRCs 2013-04-21 15:03:33 -05:00
xfs_inode_item.h
xfs_inode.c xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_inode.h xfs: add version 3 inode format with CRCs 2013-04-21 15:03:33 -05:00
xfs_inum.h
xfs_ioctl.c xfs: fallback to vmalloc for large buffers in xfs_attrlist_by_handle 2013-05-07 18:56:38 -05:00
xfs_ioctl.h
xfs_ioctl32.c xfs: fallback to vmalloc for large buffers in xfs_compat_attrlist_by_handle 2013-05-07 19:00:10 -05:00
xfs_ioctl32.h
xfs_iomap.c xfs: xfs_iomap_prealloc_size() tracepoint 2013-03-22 16:07:56 -05:00
xfs_iomap.h
xfs_iops.c
xfs_iops.h
xfs_itable.c
xfs_itable.h
xfs_linux.h xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_log_cil.c xfs: fix missing KM_NOFS tags to keep lockdep happy 2013-05-20 16:18:05 -05:00
xfs_log_priv.h xfs: Remove the obsolete XLOG_CIL_HARD_SPACE_LIMIT() macros 2013-04-16 13:18:33 -05:00
xfs_log_recover.c xfs: fix da node magic number mismatches 2013-05-01 14:48:30 -05:00
xfs_log_recover.h
xfs_log.c xfs: rename random32() to prandom_u32() 2013-03-07 12:33:57 -06:00
xfs_log.h
xfs_message.c xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_message.h xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_mount.c xfs: implement extended feature masks 2013-04-27 13:05:18 -05:00
xfs_mount.h xfs: add CRC checks to the superblock 2013-04-27 13:03:12 -05:00
xfs_mru_cache.c
xfs_mru_cache.h
xfs_qm_bhv.c
xfs_qm_syscalls.c xfs: avoid nesting transactions in xfs_qm_scall_setqlim() 2013-05-21 13:57:05 -05:00
xfs_qm.c xfs: add CRC checks for quota blocks 2013-04-21 14:58:22 -05:00
xfs_qm.h xfs: add CRC checks for quota blocks 2013-04-21 14:58:22 -05:00
xfs_quota_priv.h
xfs_quota.h xfs: add CRC checks for quota blocks 2013-04-21 14:58:22 -05:00
xfs_quotaops.c
xfs_rename.c
xfs_rtalloc.c
xfs_rtalloc.h
xfs_sb.h xfs: implement extended feature masks 2013-04-27 13:05:18 -05:00
xfs_stats.c
xfs_stats.h
xfs_super.c fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
xfs_super.h
xfs_symlink.c xfs: add CRC checks to remote symlinks 2013-04-27 11:49:28 -05:00
xfs_symlink.h xfs: add CRC checks to remote symlinks 2013-04-27 11:49:28 -05:00
xfs_sysctl.c
xfs_sysctl.h
xfs_trace.c xfs: add CRCs to dir2/da node blocks 2013-04-27 12:33:38 -05:00
xfs_trace.h xfs: xfs_iomap_prealloc_size() tracepoint 2013-03-22 16:07:56 -05:00
xfs_trans_ail.c
xfs_trans_buf.c xfs: buffer type overruns blf_flags field 2013-04-27 13:01:58 -05:00
xfs_trans_dquot.c xfs: pass xfs_dquot to xfs_qm_adjust_dqlimits() instead of xfs_disk_dquot_t 2013-03-22 16:05:52 -05:00
xfs_trans_extfree.c
xfs_trans_inode.c
xfs_trans_priv.h
xfs_trans_space.h
xfs_trans.c
xfs_trans.h xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00
xfs_types.h
xfs_utils.c
xfs_utils.h
xfs_vnode.h
xfs_vnodeops.c xfs: fix rounding in xfs_free_file_space 2013-05-20 14:25:50 -05:00
xfs_vnodeops.h
xfs_xattr.c
xfs.h xfs: introduce CONFIG_XFS_WARN 2013-05-07 18:45:36 -05:00