[gpfsug-discuss] Checking for Stale File Handles

[Mathias Dietz]

Hi Alex,

did you try mmhealth ? It should detect stale file handles of the gpfs 
filesystems already and report a "stale_mount" event. 

 
Mit freundlichen Grüßen / Kind regards

Mathias Dietz

Spectrum Scale Development - Release Lead Architect (4.2.x)
Spectrum Scale RAS Architect
---------------------------------------------------------------------------
IBM Deutschland
Am Weiher 24
65451 Kelsterbach
Phone: +49 70342744105
Mobile: +49-15152801035
E-Mail: mdietz at de.ibm.com
-----------------------------------------------------------------------------
IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Martina Koederitz, Geschäftsführung: Dirk 
WittkoppSitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht 
Stuttgart, HRB 243294



From:   Alexander John Mamach <alex.mamach at northwestern.edu>
To:     gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Cc:     "gpfsug-discuss at spectrumscale.org" 
<gpfsug-discuss at spectrumscale.org>
Date:   09/08/2019 22:33
Subject:        [EXTERNAL] Re: [gpfsug-discuss] Checking for Stale File 
Handles
Sent by:        gpfsug-discuss-bounces at spectrumscale.org



Hi Fred,
 
We sometimes find a node will show that GPFS is active when running 
mmgetstate, but one of our GPFS filesystems, (such as our home or projects 
filesystems) are inaccessible to users, while the other GPFS-mounted 
filesystems behave as expected. Our current node health checks don?t 
always detect this, especially when it?s for a resource-based mount that 
doesn?t impact the node but would impact jobs trying to run on the node.
 
If there is something native to GPFS that can detect this, all the better, 
but I?m simply unaware of how to do so.
 
Thanks,
 
Alex
 
Senior Systems Administrator

Research Computing Infrastructure
Northwestern University Information Technology (NUIT)

2020 Ridge Ave
Evanston, IL 60208-4311

O: (847) 491-2219
M: (312) 887-1881
www.it.northwestern.edu
 

From: gpfsug-discuss-bounces at spectrumscale.org 
<gpfsug-discuss-bounces at spectrumscale.org> on behalf of Frederick Stock 
<stockf at us.ibm.com>
Sent: Friday, August 9, 2019 1:03:09 PM
To: gpfsug-discuss at spectrumscale.org <gpfsug-discuss at spectrumscale.org>
Cc: gpfsug-discuss at spectrumscale.org <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Checking for Stale File Handles 
 
Are you able to explain why you want to check for stale file handles?  Are 
you attempting to detect failures of some sort, and why do the existing 
mechanisms in GPFS not provide the functionality you require?

Fred
__________________________________________________
Fred Stock | IBM Pittsburgh Lab | 720-430-8821
stockf at us.ibm.com
 
 
----- Original message -----
From: Alexander John Mamach <alex.mamach at northwestern.edu>
Sent by: gpfsug-discuss-bounces at spectrumscale.org
To: "gpfsug-discuss at spectrumscale.org" <gpfsug-discuss at spectrumscale.org>
Cc:
Subject: [EXTERNAL] [gpfsug-discuss] Checking for Stale File Handles
Date: Fri, Aug 9, 2019 1:46 PM
 
Hi folks,
 
We?re currently investigating a way to check for stale file handles on the 
nodes across our cluster in a way that minimizes impact to the filesystem 
and performance.
 
Has anyone found a direct way of doing so? We considered a few methods, 
including simply attempting to ls a GPFS filesystem from each node, but 
that might have false positives, (detecting slowdowns as stale file 
handles), and could negatively impact performance with hundreds of nodes 
doing this simultaneously.
 
Thanks,
 
Alex
 
Senior Systems Administrator

Research Computing Infrastructure
Northwestern University Information Technology (NUIT)

2020 Ridge Ave
Evanston, IL 60208-4311

O: (847) 491-2219
M: (312) 887-1881
www.it.northwestern.edu
 
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss 
 
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=9dCEbNr27klWay2AcOfvOE1xq50K-CyRUu4qQx4HOlk&m=sUjgq9g2p2ncIpALAqAhOqt7blwynTJmgmFdYYik7MI&s=EFC3lNuf6koYPMPSWuYCNhwmIMUKKZ9mCQFhxVCYWLQ&e= 





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20190812/97c122fe/attachment.htm>