Monday 4 November 2013

why du and df show different filesystem usage

Today I saw a forum post asking on his system, why df shows over 200G space used, while du only shows 50G ussed.
Most probably this is caused by open files being deleted.
When a file is opened by a process, deleting the file won't release the space occupied by it. we also need to terminate the process, otherwise df and du will show different filesystem usage.

Let's assume du and df show huge difference for /var, we can check which open files are deleted and the processes opening them.

[root@linux ~]# lsof | sed -n '1p;/var.*deleted/p'
COMMAND    PID     USER   FD      TYPE     DEVICE SIZE/OFF       NODE NAME
httpd     1779     root   10w      REG      253,0    22034       8117 /var/log/httpd/access_log (deleted)
httpd     1810   apache   10w      REG      253,0    22034       8117 /var/log/httpd/access_log (deleted)
httpd     1811   apache   10w      REG      253,0    22034       8117 /var/log/httpd/access_log (deleted)
httpd     1812   apache   10w      REG      253,0    22034       8117 /var/log/httpd/access_log (deleted)
httpd     1813   apache   10w      REG      253,0    22034       8117 /var/log/httpd/access_log (deleted)
httpd     1814   apache   10w      REG      253,0    22034       8117 /var/log/httpd/access_log (deleted)
httpd     1815   apache   10w      REG      253,0    22034       8117 /var/log/httpd/access_log (deleted)
httpd     1816   apache   10w      REG      253,0    22034       8117 /var/log/httpd/access_log (deleted)
httpd     1817   apache   10w      REG      253,0    22034       8117 /var/log/httpd/access_log (deleted)

From the output we can see that access_log is deleted, but Apache was not restarted, httpd process still has this file open.
to release the space, we can restart httpd process:
[root@linux ~]# service httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]

Now check again
[root@linux ~]# lsof | sed -n '1p;/var.*deleted/p'
COMMAND    PID     USER   FD      TYPE     DEVICE SIZE/OFF       NODE NAME

And du df report same file system usage.

Before we restart httpd process, Linux won't release the space used by access_log, if access_log is deleted by mistake, is there a way to recover it?
Yes, I will demo how to recover deleted open files in http://linuxscripter.blogspot.com/2013/11/how-to-recover-deleted-open-files.html

No comments:

Post a Comment