[gpfsug-discuss] How to prove that data is in inode
Billich Heinrich Rainer (ID SD)
heinrich.billich at id.ethz.ch
Thu Jul 18 15:15:12 BST 2019
Hello Kums,
Thank you; I could verify that the data of small files is in the inode. In the table below you see the filesize and the result of the tsdbfs query. Up to 3k all data is in the inode.
The result of stat calls on small files is very puzzling. When data is in the inode stat reports one block of 512 bytes used, even for 3k of data. I don’t see how this would affect any of our applications, so it’s just something to note.
I append some output and the script to generate it, just for completeness.
Cheers,
Heiner
# stat -c "inode %i" $f | tsdbfs fs1301 | grep indirectionLevel
0 indirectionLevel=DIRECT status=USERFILE. <empty file is special>
1 indirectionLevel=INODE status=USERFILE
16 indirectionLevel=INODE status=USERFILE
512 indirectionLevel=INODE status=USERFILE
1k indirectionLevel=INODE status=USERFILE
2k indirectionLevel=INODE status=USERFILE
3k indirectionLevel=INODE status=USERFILE. < up to 3k data is in inode >
4k indirectionLevel=DIRECT status=USERFILE
16k indirectionLevel=DIRECT status=USERFILE
64k indirectionLevel=DIRECT status=USERFILE
1M indirectionLevel=DIRECT status=USERFILE
2M indirectionLevel=DIRECT status=USERFILE
Stat output
# stat -c “%n size: %s allocated: %b*%B”
# stat -c %n size: %s allocated: %b*%B
0 size: 0 allocated: 0*512
1 size: 1 allocated: 1*512
16 size: 16 allocated: 1*512
512 size: 512 allocated: 1*512
1k size: 1024 allocated: 1*512
2k size: 2048 allocated: 1*512
3k size: 3072 allocated: 1*512 < 3k file and all data in inode: stat reports 1*512 allocated/used)
4k size: 4096 allocated: 64*512
16k size: 16384 allocated: 64*512. < as expected, 32k subblock size >
64k size: 65536 allocated: 128*512
1M size: 1048576 allocated: 2048*512
2M size: 2097152 allocated: 4096*512
The script:
# test-data-in-inode.sh
sizes="0 1 16 512 1k 2k 3k 4k 16k 64k 1M 2M"
echo create files
for s in $sizes
do
head -c $s /dev/zero > $s
done
echo sleep 20. # give gpfs some time to update metadata
sleep 20
echo
echo "# ls -ls $sizes"
ls -ls $sizes
echo
echo "# stat -c %n size: %s allocated: %b*%B"
stat -c "%n size: %s allocated: %b*%B" $sizes
echo
echo '# stat -c "inode %i" $f | tsdbfs fs1301 | grep indirectionLevel'
for f in $sizes
do
echo -n $f
stat -c "inode %i" $f | tsdbfs fs1301 | grep indirectionLevel
done
From: <gpfsug-discuss-bounces at spectrumscale.org> on behalf of Kumaran Rajaram <kums at us.ibm.com>
Reply to: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date: Wednesday, 17 July 2019 at 14:38
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] How to prove that data is in inode
Hi,
>> How can I prove that data of a small file is stored in the inode (and not on a data nsd)?
You may use echo "inode file_inode_number" | tsdbfs fs_device | grep indirectionLevel and if it points to INODE, then the file is stored in the inodes
# 4K Inode Size
# mmlsfs gpfs3a | grep 'Inode size'
-i 4096 Inode size in bytes
# Small file
# ls -l /mnt/gpfs3a/hello.txt
-rw-r--r-- 1 root root 6 Jul 17 08:32 /mnt/gpfs3a/hello.txt
# ls -i /mnt/gpfs3a/hello.txt
91649 /mnt/gpfs3a/hello.txt
#File is inlined within Inode
# echo "inode 91649" | tsdbfs gpfs3a | grep indirectionLevel
indirectionLevel=INODE status=USERFILE
Regards,
-Kums
[Inactive hide details for "Billich Heinrich Rainer (ID SD)" ---07/17/2019 07:49:56 AM---Hello, How can I prove that data of a]"Billich Heinrich Rainer (ID SD)" ---07/17/2019 07:49:56 AM---Hello, How can I prove that data of a small file is stored in the inode (and not on a data nsd)?
From: "Billich Heinrich Rainer (ID SD)" <heinrich.billich at id.ethz.ch>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date: 07/17/2019 07:49 AM
Subject: [EXTERNAL] [gpfsug-discuss] How to prove that data is in inode
Sent by: gpfsug-discuss-bounces at spectrumscale.org
________________________________
Hello,
How can I prove that data of a small file is stored in the inode (and not on a data nsd)?
We have a filesystem with 4k inodes on Scale 5.0.2 , but it seems there is no file data in the inodes?
I would expect that 'stat' reports 'Blocks: 0' for a small file, but I see 'Blocks:1'.
Cheers,
Heiner
I tried
[]# rm -f test; echo hello > test
[]# ls -ls test
1 -rw-r--r-- 1 root root 6 Jul 17 13:11 test
[root at testnas13ems01 test]# stat test
File: ‘test’
Size: 6 Blocks: 1 IO Block: 1048576 regular file
Device: 2dh/45d Inode: 353314 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2019-07-17 13:11:03.037049000 +0200
Modify: 2019-07-17 13:11:03.037331000 +0200
Change: 2019-07-17 13:11:03.037259319 +0200
Birth: -
[root at testnas13ems01 test]# du test
1 test
[root at testnas13ems01 test]# du -b test
6 test
[root at testnas13ems01 test]#
Filesystem
# mmlsfs f****
flag value description
------------------- ------------------------ -----------------------------------
-f 32768 Minimum fragment (subblock) size in bytes
-i 4096 Inode size in bytes
-I 32768 Indirect block size in bytes
-m 1 Default number of metadata replicas
-M 2 Maximum number of metadata replicas
-r 1 Default number of data replicas
-R 2 Maximum number of data replicas
-j cluster Block allocation type
-D nfs4 File locking semantics in effect
-k nfs4 ACL semantics in effect
-n 32 Estimated number of nodes that will mount file system
-B 1048576 Block size
-Q user;group;fileset Quotas accounting enabled
user;group;fileset Quotas enforced
user;group;fileset Default quotas enabled
--perfileset-quota Yes Per-fileset quota enforcement
--filesetdf Yes Fileset df enabled?
-V 20.01 (5.0.2.0) Current file system version
15.01 (4.2.0.0) Original file system version
--create-time ***** 2017 File system creation time
-z No Is DMAPI enabled?
-L 33554432 Logfile size
-E Yes Exact mtime mount option
-S relatime Suppress atime mount option
-K whenpossible Strict replica allocation option
--fastea Yes Fast external attributes enabled?
--encryption No Encryption enabled?
--inode-limit 1294592 Maximum number of inodes in all inode spaces
--log-replicas 0 Number of log replicas
--is4KAligned Yes is4KAligned?
--rapid-repair Yes rapidRepair enabled?
--write-cache-threshold 0 HAWC Threshold (max 65536)
--subblocks-per-full-block 32 Number of subblocks per full block
-P system;data Disk storage pools in file system
--file-audit-log No File Audit Logging enabled?
--maintenance-mode No Maintenance Mode enabled?
-d ******
-A yes Automatic mount option
-o nfssync,nodev Additional mount options
-T /**** Default mount point
--mount-priority 0 Mount priority
--
=======================
Heinrich Billich
ETH Zürich
Informatikdienste
Tel.: +41 44 632 72 56
heinrich.billich at id.ethz.ch
========================
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190718/d044120a/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 106 bytes
Desc: image001.gif
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190718/d044120a/attachment.gif>
More information about the gpfsug-discuss
mailing list