From l.r.sudbery at bham.ac.uk  Thu Apr  6 13:12:48 2023
From: l.r.sudbery at bham.ac.uk (Luke Sudbery)
Date: Thu, 6 Apr 2023 12:12:48 +0000
Subject: [gpfsug-discuss] mmvdisk version/communication issues?
In-Reply-To: <CWLP265MB46896CFDACE2D257A6DB6BD190819@CWLP265MB4689.GBRP265.PROD.OUTLOOK.COM>
References: <CWLP265MB4689C8BC5AD96830BFDD702890BD9@CWLP265MB4689.GBRP265.PROD.OUTLOOK.COM>
 <CWLP265MB4689098A749013BE784B67D790BD9@CWLP265MB4689.GBRP265.PROD.OUTLOOK.COM>
 <MW3PR15MB39295DB84EF8133B437563DF92819@MW3PR15MB3929.namprd15.prod.outlook.com>
 <CWLP265MB46894B8987417F01370BCEFE90819@CWLP265MB4689.GBRP265.PROD.OUTLOOK.COM>
 <MW3PR15MB392923FDB6CE5983A5916F5292819@MW3PR15MB3929.namprd15.prod.outlook.com>
 <CWLP265MB4689436D20C9B11303C2282790819@CWLP265MB4689.GBRP265.PROD.OUTLOOK.COM>
 <2A15B005-B65A-46F0-8218-3A0275F70166@rutgers.edu>
 <CWLP265MB46896CFDACE2D257A6DB6BD190819@CWLP265MB4689.GBRP265.PROD.OUTLOOK.COM>
Message-ID: <CWLP265MB46893ADA1775228B084181F690919@CWLP265MB4689.GBRP265.PROD.OUTLOOK.COM>

Just to close the loop on this, if anyone is having similar problems and wants to reference the cases:

IBM created efix 37 for 5.1.5.1 for this as g515p1e37.x86_64-linux-gnr.tar.gz under TS012531429.

Lenovo have a copy of this and have provided update instructions for the DSSG under case # 1001583845 (basically fail the recovery groups over and install the updated RPMs).

And this all appears to be working well for us now, so I can proceed with updating the other building blocks. A new version is due which will probably come out while I?m in the middle of doing that of course?

Many thanks,

Luke

--
Luke Sudbery
Principal Engineer (HPC and Storage).
Architecture, Infrastructure and Systems
Advanced Research Computing, IT Services
Room 132, Computer Centre G5, Elms Road

Please note I don?t work on Monday.

From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> On Behalf Of Luke Sudbery
Sent: 21 March 2023 13:47
To: gpfsug main discussion list <gpfsug-discuss at gpfsug.org>
Subject: Re: [gpfsug-discuss] mmvdisk version/communication issues?

Yes, that?s exactly what happened to me. If Lenovo had tested an upgrade they would have seen it too.

We have 3 DSSG, but they are all this cluster, and we don?t run GNR outside this cluster.

The 5.1.5.1 nodes can talk to each other OK. And to the downlevel nodes. But the downlevel nodes can?t talk to the newer nodes.

At present it doesn?t seem to present any major problem ? but we haven?t had to move RGs around or do anything like that yet.

And we have a new DSSG to add to this cluster soon, and then some pool migrations. This needs to sorted before we can start that in confidence.

Many thanks,

Luke

--
Luke Sudbery
Principal Engineer (HPC and Storage).
Architecture, Infrastructure and Systems
Advanced Research Computing, IT Services
Room 132, Computer Centre G5, Elms Road

Please note I don?t work on Monday.

From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org<mailto:gpfsug-discuss-bounces at gpfsug.org>> On Behalf Of Ryan Novosielski
Sent: 21 March 2023 13:33
To: gpfsug main discussion list <gpfsug-discuss at gpfsug.org<mailto:gpfsug-discuss at gpfsug.org>>
Subject: Re: [gpfsug-discuss] mmvdisk version/communication issues?

CAUTION: This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.

You know, I ran into this during the upgrade, didn?t have an explanation for it, and decided that I would see what happened when it was finished (problem was gone). I was going from DSS-G 3.2a to DSS-G 4.3a. It was at the stage where I had one machine upgraded and one machine not upgraded, which makes sense. I believe I then tried it from the upgraded node, it worked fine, and I figured ?well, I?m blowing that one away shortly anyway.?

Would one expect to only see this while nodes in the same cluster are not running the same version? I guess what I?m asking is would one expect to have that kind of traffic coming from any place else except the same cluster?

Sent from my iPhone

On Mar 21, 2023, at 08:38, Luke Sudbery <L.R.Sudbery at bham.ac.uk<mailto:L.R.Sudbery at bham.ac.uk>> wrote:
?
Thank you, I?ll pass that onto Lenovo. I think I?ve also identified it as this in the fixies in 5.1.6.1 too:
https://www.ibm.com/support/pages/apar/IJ44607
IJ44607: SPECTRUM SCALE V5.1.3 AND GREATER FAILS TO INTEROPERATE WITH SPECTRUMSCALE VERSIONS PRIOR TO 5.1.3.
Problem summary

1.  GNR RPCs fail when received by a GPFS daemon 5.1.3 or later

2.  from a GPFS daemon older than version 5.1.3.

3.  Kernel assert going off: privVfsP != NULL

Symptom:

1.  Hang in the command

As such, AFAICT it will affect all Lenovo customers going from DSSG 3.x (GPFS 5.1.1.0) to 4.x (GPFS 5.1.5.1 efix20) and I?m a bit annoyed that didn?t pick it up before release.
Many thanks,

Luke

--
Luke Sudbery
Principal Engineer (HPC and Storage).
Architecture, Infrastructure and Systems
Advanced Research Computing, IT Services
Room 132, Computer Centre G5, Elms Road

Please note I don?t work on Monday.

From: Gang Qiu <gangqiu at cn.ibm.com<mailto:gangqiu at cn.ibm.com>>
Sent: 21 March 2023 12:31
To: Luke Sudbery (Advanced Research Computing) <l.r.sudbery at bham.ac.uk<mailto:l.r.sudbery at bham.ac.uk>>; gpfsug main discussion list <gpfsug-discuss at gpfsug.org<mailto:gpfsug-discuss at gpfsug.org>>
Subject: ??: mmvdisk version/communication issues?

CAUTION: This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.



Luke,

1. The case number is TS011014198 (Internal defect number is 1163600)
   5.1.3.1 efix47 includes this fix. (I didn't find the efix for 5.1.4 or 5.1.5)
2. The impact is that any GNR-related command from 5.1.2 to 5.1.5 will hang.

Regards,
Gang Qiu
====================================================
Gang Qiu(??)
Spectrum Scale Development - ECE
IBM China Systems Lab
Mobile: +86-18612867902
====================================================


???: Luke Sudbery <l.r.sudbery at bham.ac.uk<mailto:l.r.sudbery at bham.ac.uk>>
??: ???, 2023?3?21? 17:31
???: gpfsug main discussion list <gpfsug-discuss at gpfsug.org<mailto:gpfsug-discuss at gpfsug.org>>, Gang Qiu <gangqiu at cn.ibm.com<mailto:gangqiu at cn.ibm.com>>
??: [EXTERNAL] RE: mmvdisk version/communication issues?
Thank you, that?s far more useful than Lenovo support have been so far! Unfortunately Lenovo only support particular versions of scale on their DSSG and they are our only source of the required GNR packages. So a couple more questions if you
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd
Thank you, that?s far more useful than Lenovo support have been so far!

Unfortunately Lenovo only support particular versions of scale on their DSSG and they are our only source of the required GNR packages.

So a couple more questions if you don?t mind!


  1.  Can you provide a link the APAR or similar I can share with Lenovo?
  2.  Do you know of any workaround or other impact of this issue? (I think I?ve seen mmheath show false errors because of it). This may help us to decide whether to just press ahead upgrading to 5.1.5 everywhere if the issue is not present 5.1.5 -> 5.1.5.

Many thanks,

Luke


--
Luke Sudbery
Principal Engineer (HPC and Storage).
Architecture, Infrastructure and Systems
Advanced Research Computing, IT Services
Room 132, Computer Centre G5, Elms Road

Please note I don?t work on Monday.

From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org<mailto:gpfsug-discuss-bounces at gpfsug.org>> On Behalf Of Gang Qiu
Sent: 21 March 2023 03:49
To: gpfsug main discussion list <gpfsug-discuss at gpfsug.org<mailto:gpfsug-discuss at gpfsug.org>>
Subject: [gpfsug-discuss] ??: mmvdisk version/communication issues?

CAUTION: This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.




It is not mmvdisk issue. Mmvdisk hangs because ?tslsrecgroup? command hangs which executed by mmvdisk.
I recreated it on my environment (coexist 5.1.3 or 5.1.4, 5.1.5 with 5.1.2 or previous version)

After further investigation, it should be a known issue which is fixed in 5.1.6 PTF1.
To resolve this issue, we need to upgrade the version to 5.1.6.1 at least.


Regards,
Gang Qiu
====================================================
Gang Qiu(??)
Spectrum Scale Development - ECE
IBM China Systems Lab
Mobile: +86-18612867902
====================================================


???: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org<mailto:gpfsug-discuss-bounces at gpfsug.org>> ?? Luke Sudbery <l.r.sudbery at bham.ac.uk<mailto:l.r.sudbery at bham.ac.uk>>
??: ???, 2023?3?18? 00:06
???: gpfsug main discussion list <gpfsug-discuss at gpfsug.org<mailto:gpfsug-discuss at gpfsug.org>>
??: [EXTERNAL] Re: [gpfsug-discuss] mmvdisk version/communication issues?
On further investigation the command does eventually complete, after 11 minutes rather than a couple of seconds. [root@?rds-pg-dssg01 ~]# time mmvdisk pdisk list --rg rds_er_dssg02 --not-ok mmvdisk: All pdisks of recovery group 'rds_er_dssg02'
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd
On further investigation the command does eventually complete, after 11 minutes rather than a couple of seconds.


[root at rds-pg-dssg01 ~]# time mmvdisk pdisk list --rg rds_er_dssg02 --not-ok

mmvdisk: All pdisks of recovery group 'rds_er_dssg02' are ok.



real    11m14.106s

user    0m1.430s

sys     0m0.555s

[root at rds-pg-dssg01 ~]#

Looking at the process tree, the bits that hang are:

[root at rds-pg-dssg01 ~]# time tslsrecgroup rds_er_dssg02 -Y --v2 --failure-domain

Failed to connect to file system daemon: Connection timed out



real    5m30.181s

user    0m0.001s

sys     0m0.003s

and then

[root at rds-pg-dssg01 ~]# time tslspdisk --recovery-group rds_er_dssg02 --notOK

Failed to connect to file system daemon: Connection timed out



real    5m30.247s

user    0m0.003s

sys     0m0.002s

[root at rds-pg-dssg01 ~]#

Which adds up to the 11 minutes.... then it does something else and just works. Or maybe it doesn't work and just wouldn't report any failed disks is there were any?.

While hanging, the ts commands appear to be LISTENing, not attempting to make connections:


[root at rds-pg-dssg01 ~]# pidof tslspdisk

2156809

[root at rds-pg-dssg01 ~]# netstat -apt | grep 2156809

tcp        0      0 0.0.0.0:60000           0.0.0.0:*               LISTEN      2156809/tslspdisk

[root at rds-pg-dssg01 ~]#

Port 60000 is the lowest of our tscCmdPortRange.

Don?t know if that helps anyone?.

Cheers,

Luke
--
Luke Sudbery
Principal Engineer (HPC and Storage).
Architecture, Infrastructure and Systems
Advanced Research Computing, IT Services
Room 132, Computer Centre G5, Elms Road

Please note I don?t work on Monday.

From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org<mailto:gpfsug-discuss-bounces at gpfsug.org>> On Behalf Of Luke Sudbery
Sent: 17 March 2023 15:11
To: gpfsug-discuss at gpfsug.org<mailto:gpfsug-discuss at gpfsug.org>
Subject: [gpfsug-discuss] mmvdisk version/communication issues?

Hello,

We 3 Lenovo DSSG ?Building Blocks? as they call them ? 2x GNR server pairs.

We?ve just upgraded the 1st of them from 3.2a (GPFS 5.1.1.0) to 4.3a (5.1.5.1 efix 20).

Now the older systems can?t communicated with the newer in certain circumstances, specifically querying recovery groups hosted on other servers.

It works old->old, new->old and new->new but not old->new.

If fairly sure it is not a TCP comms problem. I can ssh between the node as root and as the GPFS sudoUser. Port 1191 and the tscCmdPortRange are open and accessible in both direction between the nodes. There are connections present between the nodes in netstat and in mmfsd.latest.log. No pending message (to that node) in mmdiag --network.

In these examples rds-er-dssg01/2 are upgraded, rds-pg-dssg01/2 are downlevel:


[root at rds-er-dssg01 ~]# mmvdisk pdisk list --rg rds_er_dssg02 --not-ok  # New to new

mmvdisk: All pdisks of recovery group 'rds_er_dssg02' are ok.

[root at rds-er-dssg01 ~]# mmvdisk pdisk list --rg rds_pg_dssg02 --not-ok  # New to old

mmvdisk: All pdisks of recovery group 'rds_pg_dssg02' are ok.

[root at rds-er-dssg01 ~]#


[root at rds-pg-dssg01 ~]# mmvdisk pdisk list --rg rds_pg_dssg02 --not-ok # Old to old

mmvdisk: All pdisks of recovery group 'rds_pg_dssg02' are ok.

[root at rds-pg-dssg01 ~]# mmvdisk pdisk list --rg rds_er_dssg02 --not-ok # Old to new [HANGS]

^Cmmvdisk: Command failed. Examine previous error messages to determine cause.

[root at rds-pg-dssg01 ~]#

Has anyone come across this? mmvdisk should work across slightly different versions of 5.1, right? No recovery group, cluster or filesystem versions have been changed yet.

We will also log a ticket with snaps and more info but wondered if anyone had seen this.

And while this particular command is not a major issue, we don?t know what else it may affect, before we proceed with the reset of the cluster.

Many thanks,

Luke

--
Luke Sudbery
Principal Engineer (HPC and Storage).
Architecture, Infrastructure and Systems
Advanced Research Computing, IT Services
Room 132, Computer Centre G5, Elms Road

Please note I don?t work on Monday.

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230406/0e57497c/attachment.htm>

From alvise.dorigo at psi.ch  Thu Apr 13 10:15:46 2023
From: alvise.dorigo at psi.ch (Dorigo Alvise)
Date: Thu, 13 Apr 2023 09:15:46 +0000
Subject: [gpfsug-discuss] GPFS UK Meeting Wednesday 21st June - Thursday
 22nd June 2023 - Postponed
In-Reply-To: <852d6fde37de92b95917092171e1a008@gpfsug.org>
References: <b104eef220a0ae251af31c730fc416d0@gpfsug.org>
 <852d6fde37de92b95917092171e1a008@gpfsug.org>
Message-ID: <ZR0P278MB0107E1D82A33CFE392E6C90287989@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>

Good Morning,

is there at least an idea about the month when the UK user group meeting will be held ?

Thanks a lot,

   Alvise Dorigo,
   Paul Scherrer Institut
   Switzerland
________________________________
From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> on behalf of chair at gpfsug.org <chair at gpfsug.org>
Sent: Friday, March 3, 2023 12:43 PM
To: Gpfsug Discuss <gpfsug-discuss at spectrumscaleug.org>
Subject: Re: [gpfsug-discuss] GPFS UK Meeting Wednesday 21st June - Thursday 22nd June 2023 - Postponed

Dear All,

Due to an issue with the IBM Centre, we have had to postpone the planned
meeting in June and will update you with a new date in the near future

Sorry for this

Regards

Paul

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230413/82e1c5a9/attachment.htm>

From alvise.dorigo at psi.ch  Thu Apr 13 10:15:46 2023
From: alvise.dorigo at psi.ch (Dorigo Alvise)
Date: Thu, 13 Apr 2023 09:15:46 +0000
Subject: [gpfsug-discuss] GPFS UK Meeting Wednesday 21st June - Thursday
 22nd June 2023 - Postponed
In-Reply-To: <852d6fde37de92b95917092171e1a008@gpfsug.org>
References: <b104eef220a0ae251af31c730fc416d0@gpfsug.org>
 <852d6fde37de92b95917092171e1a008@gpfsug.org>
Message-ID: <ZR0P278MB0107E1D82A33CFE392E6C90287989@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>

Good Morning,

is there at least an idea about the month when the UK user group meeting will be held ?

Thanks a lot,

   Alvise Dorigo,
   Paul Scherrer Institut
   Switzerland
________________________________
From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> on behalf of chair at gpfsug.org <chair at gpfsug.org>
Sent: Friday, March 3, 2023 12:43 PM
To: Gpfsug Discuss <gpfsug-discuss at spectrumscaleug.org>
Subject: Re: [gpfsug-discuss] GPFS UK Meeting Wednesday 21st June - Thursday 22nd June 2023 - Postponed

Dear All,

Due to an issue with the IBM Centre, we have had to postpone the planned
meeting in June and will update you with a new date in the near future

Sorry for this

Regards

Paul

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230413/82e1c5a9/attachment-0001.htm>

From Paul.Tomlinson at awe.co.uk  Thu Apr 13 10:21:07 2023
From: Paul.Tomlinson at awe.co.uk (Tomlinson Paul J (DS&T) AWE)
Date: Thu, 13 Apr 2023 09:21:07 +0000
Subject: [gpfsug-discuss] EXTERNAL: Re: GPFS UK Meeting Wednesday 21st
 June - Thursday 22nd June 2023 - Postponed
In-Reply-To: <ZR0P278MB0107E1D82A33CFE392E6C90287989@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>
References: <b104eef220a0ae251af31c730fc416d0@gpfsug.org>
 <852d6fde37de92b95917092171e1a008@gpfsug.org>
 <ZR0P278MB0107E1D82A33CFE392E6C90287989@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>
Message-ID: <LO2P123MB57439E4D880C823CE6B541C7D2989@LO2P123MB5743.GBRP123.PROD.OUTLOOK.COM>

Hi all,

The planned date is currently Tuesday 27th and Wednesday 28th July.

I will send out an email later today

Regards

Paul

Paul Tomlinson

From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> On Behalf Of Dorigo Alvise
Sent: 13 April 2023 10:16
To: Gpfsug Discuss <gpfsug-discuss at spectrumscaleug.org>; gpfsug main discussion list <gpfsug-discuss at gpfsug.org>
Subject: EXTERNAL: Re: [gpfsug-discuss] GPFS UK Meeting Wednesday 21st June - Thursday 22nd June 2023 - Postponed


PHISHING ALERT - This email originated from outside of AWE
DO YOU TRUST THIS EMAIL? If not, click here<https://aweuk.sharepoint.com/teams/CRIA/CyberOps/CyberSecurity/SitePages/The-PhishTank.aspx> for more guidance or use the 'Report Message' button on your Outlook toolbar to send this message to the PhishTank.
Good Morning,

is there at least an idea about the month when the UK user group meeting will be held ?

Thanks a lot,

   Alvise Dorigo,
   Paul Scherrer Institut
   Switzerland
________________________________
From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> on behalf of chair at gpfsug.org <chair at gpfsug.org>
Sent: Friday, March 3, 2023 12:43 PM
To: Gpfsug Discuss <gpfsug-discuss at spectrumscaleug.org>
Subject: Re: [gpfsug-discuss] GPFS UK Meeting Wednesday 21st June - Thursday 22nd June 2023 - Postponed

Dear All,

Due to an issue with the IBM Centre, we have had to postpone the planned
meeting in June and will update you with a new date in the near future

Sorry for this

Regards

Paul

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
The information in this email and in any attachment(s) is commercial in confidence. If you are not the named addressee(s) or if you receive this email in error then any distribution, copying or use of this communication or the information in it is strictly prohibited. Please notify us immediately by email at admin.internet(at)awe.co.uk, and then delete this message from your computer. While attachments are virus checked, AWE plc does not accept any liability in respect of any virus which is not detected. AWE Plc Registered in England and Wales Registration No 02763902 AWE, Aldermaston, Reading, RG7 4PR
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230413/11162712/attachment.htm>

From Paul.Tomlinson at awe.co.uk  Thu Apr 13 10:21:07 2023
From: Paul.Tomlinson at awe.co.uk (Tomlinson Paul J (DS&T) AWE)
Date: Thu, 13 Apr 2023 09:21:07 +0000
Subject: [gpfsug-discuss] EXTERNAL: Re: GPFS UK Meeting Wednesday 21st
 June - Thursday 22nd June 2023 - Postponed
In-Reply-To: <ZR0P278MB0107E1D82A33CFE392E6C90287989@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>
References: <b104eef220a0ae251af31c730fc416d0@gpfsug.org>
 <852d6fde37de92b95917092171e1a008@gpfsug.org>
 <ZR0P278MB0107E1D82A33CFE392E6C90287989@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>
Message-ID: <LO2P123MB57439E4D880C823CE6B541C7D2989@LO2P123MB5743.GBRP123.PROD.OUTLOOK.COM>

Hi all,

The planned date is currently Tuesday 27th and Wednesday 28th July.

I will send out an email later today

Regards

Paul

Paul Tomlinson

From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> On Behalf Of Dorigo Alvise
Sent: 13 April 2023 10:16
To: Gpfsug Discuss <gpfsug-discuss at spectrumscaleug.org>; gpfsug main discussion list <gpfsug-discuss at gpfsug.org>
Subject: EXTERNAL: Re: [gpfsug-discuss] GPFS UK Meeting Wednesday 21st June - Thursday 22nd June 2023 - Postponed


PHISHING ALERT - This email originated from outside of AWE
DO YOU TRUST THIS EMAIL? If not, click here<https://aweuk.sharepoint.com/teams/CRIA/CyberOps/CyberSecurity/SitePages/The-PhishTank.aspx> for more guidance or use the 'Report Message' button on your Outlook toolbar to send this message to the PhishTank.
Good Morning,

is there at least an idea about the month when the UK user group meeting will be held ?

Thanks a lot,

   Alvise Dorigo,
   Paul Scherrer Institut
   Switzerland
________________________________
From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> on behalf of chair at gpfsug.org <chair at gpfsug.org>
Sent: Friday, March 3, 2023 12:43 PM
To: Gpfsug Discuss <gpfsug-discuss at spectrumscaleug.org>
Subject: Re: [gpfsug-discuss] GPFS UK Meeting Wednesday 21st June - Thursday 22nd June 2023 - Postponed

Dear All,

Due to an issue with the IBM Centre, we have had to postpone the planned
meeting in June and will update you with a new date in the near future

Sorry for this

Regards

Paul

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
The information in this email and in any attachment(s) is commercial in confidence. If you are not the named addressee(s) or if you receive this email in error then any distribution, copying or use of this communication or the information in it is strictly prohibited. Please notify us immediately by email at admin.internet(at)awe.co.uk, and then delete this message from your computer. While attachments are virus checked, AWE plc does not accept any liability in respect of any virus which is not detected. AWE Plc Registered in England and Wales Registration No 02763902 AWE, Aldermaston, Reading, RG7 4PR
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230413/11162712/attachment-0001.htm>

From sthompson2 at lenovo.com  Thu Apr 13 10:31:40 2023
From: sthompson2 at lenovo.com (Simon Thompson2)
Date: Thu, 13 Apr 2023 09:31:40 +0000
Subject: [gpfsug-discuss] [External] Re: EXTERNAL: Re: GPFS UK Meeting
 Wednesday 21st June - Thursday 22nd June 2023 - Postponed
In-Reply-To: <LO2P123MB57439E4D880C823CE6B541C7D2989@LO2P123MB5743.GBRP123.PROD.OUTLOOK.COM>
References: <b104eef220a0ae251af31c730fc416d0@gpfsug.org>
 <852d6fde37de92b95917092171e1a008@gpfsug.org>
 <ZR0P278MB0107E1D82A33CFE392E6C90287989@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>
 <LO2P123MB57439E4D880C823CE6B541C7D2989@LO2P123MB5743.GBRP123.PROD.OUTLOOK.COM>
Message-ID: <SG2PR03MB29427DDE1D3A32504CCDBA06F9989@SG2PR03MB2942.apcprd03.prod.outlook.com>


Is that Tue 27th/Wed 28th June or Thu 27th/Fri 28th July?

Thanks

Simon
________________________________
Simon Thompson
He/Him/His
Sr. Manager, HPC Storage and Performance
Chineham Business Park,
Crockford Lane, Basingstoke, Hampshire, RG24 8WQ
HPC Customer Solutions
Lenovo UK
[Phone]+44 7788 320635
[Email]sthompson2 at lenovo.com<mailto:sthompson2 at lenovo.com>

Lenovo.com/uk<http://www.lenovo.com/uk>
Twitter<http://twitter.com/lenovo> | Instagram<https://instagram.com/lenovo> | Facebook<http://www.facebook.com/lenovo> | Linkedin<http://www.linkedin.com/company/lenovo> | YouTube<http://www.youtube.com/lenovovision> | Privacy<https://www.lenovo.com/gb/en/privacy-selector/>
[cid:image003.png at 01D96DF2.413AB520]


From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> On Behalf Of Tomlinson Paul J (DS&T) AWE
Sent: 13 April 2023 10:21
To: gpfsug main discussion list <gpfsug-discuss at gpfsug.org>; Gpfsug Discuss <gpfsug-discuss at spectrumscaleug.org>
Subject: [External] Re: [gpfsug-discuss] EXTERNAL: Re: GPFS UK Meeting Wednesday 21st June - Thursday 22nd June 2023 - Postponed

Hi all,

The planned date is currently Tuesday 27th and Wednesday 28th July.

I will send out an email later today

Regards

Paul

Paul Tomlinson

From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org<mailto:gpfsug-discuss-bounces at gpfsug.org>> On Behalf Of Dorigo Alvise
Sent: 13 April 2023 10:16
To: Gpfsug Discuss <gpfsug-discuss at spectrumscaleug.org<mailto:gpfsug-discuss at spectrumscaleug.org>>; gpfsug main discussion list <gpfsug-discuss at gpfsug.org<mailto:gpfsug-discuss at gpfsug.org>>
Subject: EXTERNAL: Re: [gpfsug-discuss] GPFS UK Meeting Wednesday 21st June - Thursday 22nd June 2023 - Postponed


PHISHING ALERT - This email originated from outside of AWE
DO YOU TRUST THIS EMAIL? If not, click here<https://aweuk.sharepoint.com/teams/CRIA/CyberOps/CyberSecurity/SitePages/The-PhishTank.aspx> for more guidance or use the 'Report Message' button on your Outlook toolbar to send this message to the PhishTank.
Good Morning,

is there at least an idea about the month when the UK user group meeting will be held ?

Thanks a lot,

   Alvise Dorigo,
   Paul Scherrer Institut
   Switzerland
________________________________
From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org<mailto:gpfsug-discuss-bounces at gpfsug.org>> on behalf of chair at gpfsug.org<mailto:chair at gpfsug.org> <chair at gpfsug.org<mailto:chair at gpfsug.org>>
Sent: Friday, March 3, 2023 12:43 PM
To: Gpfsug Discuss <gpfsug-discuss at spectrumscaleug.org<mailto:gpfsug-discuss at spectrumscaleug.org>>
Subject: Re: [gpfsug-discuss] GPFS UK Meeting Wednesday 21st June - Thursday 22nd June 2023 - Postponed

Dear All,

Due to an issue with the IBM Centre, we have had to postpone the planned
meeting in June and will update you with a new date in the near future

Sorry for this

Regards

Paul

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
The information in this email and in any attachment(s) is commercial in confidence. If you are not the named addressee(s) or if you receive this email in error then any distribution, copying or use of this communication or the information in it is strictly prohibited. Please notify us immediately by email at admin.internet(at)awe.co.uk, and then delete this message from your computer. While attachments are virus checked, AWE plc does not accept any liability in respect of any virus which is not detected. AWE Plc Registered in England and Wales Registration No 02763902 AWE, Aldermaston, Reading, RG7 4PR
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230413/d9f854de/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 92 bytes
Desc: image001.gif
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230413/d9f854de/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.gif
Type: image/gif
Size: 128 bytes
Desc: image002.gif
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230413/d9f854de/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 9878 bytes
Desc: image003.png
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230413/d9f854de/attachment.png>

From chair at gpfsug.org  Sun Apr 16 09:07:25 2023
From: chair at gpfsug.org (chair at gpfsug.org)
Date: Sun, 16 Apr 2023 09:07:25 +0100
Subject: [gpfsug-discuss] GPFS UK Meeting Tuesday 27th June - Wednesday 28th
 June 2023
Message-ID: <58b6f4c09c4ab45c30d43401237f8cb6@gpfsug.org>

Hi all,

Just a reminder that the next UK User Group meeting will be taking place 
in London (IBM York Road) on Tuesday 27th and Wednesday 28th June

Now is your opportunity to help shape the agenda, please feel free to 
send me ideas for talks we could ask IBM for (I don't promise we'll get 
them on the agenda mind!).

Also if you would like to do a user talk, please get in touch with me 
and let me know. It could be a large scale deployment or even just a 
couple of nodes, it's your opportunity to showcase how you use Spectrum 
Scale and what for. Every year people tell us how valuable they find the 
user talks, but this needs YOU, so please do think about if you are able 
to offer a talk!

As in the past we are looking for sponsorship to enable us to run an 
evening networking event. I've sent out details to those who have 
sponsored in the past and to those who have asked us directly about 
sponsorship opportunities. If you either haven't received this or are 
interested in becoming a sponsor, please email me directly.

Thanks

Paul



From TROPPENS at de.ibm.com  Wed Apr 19 13:42:17 2023
From: TROPPENS at de.ibm.com (Ulf Troppens)
Date: Wed, 19 Apr 2023 12:42:17 +0000
Subject: [gpfsug-discuss] Storage Scale User Meeting at ISC 2023
Message-ID: <BYAPR15MB2983C81A89D4FA95965675A6ED629@BYAPR15MB2983.namprd15.prod.outlook.com>

Greetings, IBM is organizing a Storage Scale User Meeting at ISC 2023. We have an exciting agenda covering user stories, roadmap update, insights into potential future product enhancements, plus access to IBM experts and your peers. We look forward to welcoming you to this event. The user meeting is followed by a Get Together to continue the discussion.

Date: Mon, 22. May 2023 - 13:00-17:00 CET Local time | Hotel Reichshof

Please check the event page for agenda details and registration:
https://www.spectrumscaleug.org/event/storage-scale-user-meeting-at-isc-2023/

Ulf Troppens
Senior Technical Staff Member
Spectrum Scale Development

IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Gregor Pillen / Gesch?ftsf?hrung: David Faller
Sitz der Gesellschaft: B?blingen / Registergericht: Amtsgericht Stuttgart, HRB 243294

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230419/e17507f6/attachment.htm>

From alvise.dorigo at psi.ch  Thu Apr 20 13:32:24 2023
From: alvise.dorigo at psi.ch (Dorigo Alvise)
Date: Thu, 20 Apr 2023 12:32:24 +0000
Subject: [gpfsug-discuss] mmvdisk and --mmcrfs option
Message-ID: <ZR0P278MB01072D5CDCF20AF773EE1BB287639@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>

Hello everyone,

I read in the documentation that --mmcrfs option of mmvdisk command is conceived to pass to the latter options degined for the former (mmcrfs).
But it seems that is doesn't work for the specific command:

mmvdisk filesystem create ..... --mmcrfs --version <string>

Something really silly that I am doing ? Is it possible to create on recent GPFS versions, filesystem with old versions?

Thanks,

   Alvise
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230420/d82a1223/attachment.htm>

From gangqiu at cn.ibm.com  Thu Apr 20 13:44:58 2023
From: gangqiu at cn.ibm.com (Gang Qiu)
Date: Thu, 20 Apr 2023 12:44:58 +0000
Subject: [gpfsug-discuss] =?gb2312?b?tPC4tDogbW12ZGlzayBhbmQgLS1tbWNyZnMg?=
 =?gb2312?b?b3B0aW9u?=
In-Reply-To: <ZR0P278MB01072D5CDCF20AF773EE1BB287639@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>
References: <ZR0P278MB01072D5CDCF20AF773EE1BB287639@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>
Message-ID: <MW3PR15MB392974942A86B9C537C09A1392639@MW3PR15MB3929.namprd15.prod.outlook.com>


Alvise,

?--version {version string}? is supported by ?--mmcrfs? option. mmvdisk pass it to ?mmcrfs? directly.
For example:
/usr/lpp/mmfs/bin/mmvdisk filesystem create --fs {fsname} --vs {vs name} --mmcrfs --version 5.0.5.1

In this sample, ?--version 5.0.5.1? is pass to ?mmcrfs?

Regards,
Gang Qiu
====================================================
Gang Qiu(??)
Spectrum Scale Development - ECE
IBM China Systems Lab
====================================================


???: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> ?? Dorigo Alvise <alvise.dorigo at psi.ch>
??: ???, 2023?4?20? 20:34
???: gpfsug-discuss at gpfsug.org <gpfsug-discuss at gpfsug.org>
??: [EXTERNAL] [gpfsug-discuss] mmvdisk and --mmcrfs option
Hello everyone, I read in the documentation that --mmcrfs option of mmvdisk command is conceived to pass to the latter options degined for the former (mmcrfs). But it seems that is doesn't work for the specific command: mmvdisk filesystem create
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd
Hello everyone,

I read in the documentation that --mmcrfs option of mmvdisk command is conceived to pass to the latter options degined for the former (mmcrfs).
But it seems that is doesn't work for the specific command:

mmvdisk filesystem create ..... --mmcrfs --version <string>

Something really silly that I am doing ? Is it possible to create on recent GPFS versions, filesystem with old versions?

Thanks,

   Alvise
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230420/04f3d252/attachment.htm>

From alvise.dorigo at psi.ch  Thu Apr 20 14:05:22 2023
From: alvise.dorigo at psi.ch (Dorigo Alvise)
Date: Thu, 20 Apr 2023 13:05:22 +0000
Subject: [gpfsug-discuss] mmvdisk and --mmcrfs option
In-Reply-To: <MW3PR15MB392974942A86B9C537C09A1392639@MW3PR15MB3929.namprd15.prod.outlook.com>
References: <ZR0P278MB01072D5CDCF20AF773EE1BB287639@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>
 <MW3PR15MB392974942A86B9C537C09A1392639@MW3PR15MB3929.namprd15.prod.outlook.com>
Message-ID: <ZR0P278MB0107BF90D6F907A0BE4FFB9487639@ZR0P278MB0107.CHEP278.PROD.OUTLOOK.COM>

it worked, thank you!
I do not know why, but I was instisting to pass the FS version, instead of GPFS version... also the error reporting of the command is misleading (because it says "--version option unknown" instead of "23.00 value of --version unsupported", or something like that).

Thanks,

   Alvise
________________________________
From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> on behalf of Gang Qiu <gangqiu at cn.ibm.com>
Sent: Thursday, April 20, 2023 2:44 PM
To: gpfsug main discussion list <gpfsug-discuss at gpfsug.org>
Subject: [gpfsug-discuss] ??: mmvdisk and --mmcrfs option




Alvise,



?--version {version string}? is supported by ?--mmcrfs? option. mmvdisk pass it to ?mmcrfs? directly.

For example:

/usr/lpp/mmfs/bin/mmvdisk filesystem create --fs {fsname} --vs {vs name} --mmcrfs --version 5.0.5.1



In this sample, ?--version 5.0.5.1? is pass to ?mmcrfs?



Regards,

Gang Qiu

====================================================

Gang Qiu(??)

Spectrum Scale Development - ECE

IBM China Systems Lab

====================================================





???: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> ?? Dorigo Alvise <alvise.dorigo at psi.ch>
??: ???, 2023?4?20? 20:34
???: gpfsug-discuss at gpfsug.org <gpfsug-discuss at gpfsug.org>
??: [EXTERNAL] [gpfsug-discuss] mmvdisk and --mmcrfs option

Hello everyone, I read in the documentation that --mmcrfs option of mmvdisk command is conceived to pass to the latter options degined for the former (mmcrfs). But it seems that is doesn't work for the specific command: mmvdisk filesystem create

ZjQcmQRYFpfptBannerStart

This Message Is From an External Sender

This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd

Hello everyone,



I read in the documentation that --mmcrfs option of mmvdisk command is conceived to pass to the latter options degined for the former (mmcrfs).

But it seems that is doesn't work for the specific command:



mmvdisk filesystem create ..... --mmcrfs --version <string>

Something really silly that I am doing ? Is it possible to create on recent GPFS versions, filesystem with old versions?

Thanks,



   Alvise
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230420/f2ef6b84/attachment.htm>

From p.ward at nhm.ac.uk  Tue Apr 25 12:55:59 2023
From: p.ward at nhm.ac.uk (Paul Ward)
Date: Tue, 25 Apr 2023 11:55:59 +0000
Subject: [gpfsug-discuss] [EXTERNAL]  mmbackup vs SOBAR
In-Reply-To: <DU0PR07MB85133C76E540B16DC4D2D59DA4B59@DU0PR07MB8513.eurprd07.prod.outlook.com>
References: <LO4P265MB4050C9CC3B577FBFA7B1CAFDACB59@LO4P265MB4050.GBRP265.PROD.OUTLOOK.COM>
 <DU0PR07MB85133C76E540B16DC4D2D59DA4B59@DU0PR07MB8513.eurprd07.prod.outlook.com>
Message-ID: <LO0P265MB5215BCD77EB158E3456030F6D3649@LO0P265MB5215.GBRP265.PROD.OUTLOOK.COM>

Hi Peter and Robert,

Sorry for the delayed reply, only occasionally check the mailing list.
I'm happy to have a MS Teams (or other platform) call about the setup you are talking about, as we've just decommission that kind of environment.

Spectrum SCALE
Spectrum Protect with HSM using a dual robot High density tape library with off site copy
SOBAR - (in theory) implemented.

Kindest regards,
Paul

Paul Ward
TS Infrastructure Architect
Natural History Museum
T: 02079426450
E: p.ward at nhm.ac.uk


-----Original Message-----
From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> On Behalf Of Peter Childs
Sent: Thursday, March 9, 2023 4:08 PM
To: gpfsug-discuss at gpfsug.org
Subject: Re: [gpfsug-discuss] [EXTERNAL] mmbackup vs SOBAR

I've been told "you really should be using SOBAR" a few times, but never really understood how to do so and the steps involved. I feel sure it should have some kind of white paper, so far I've been thinking to setup some kind of test system, but get a little lost on where to start (and lack of time).

We currently use mmbackup to x2 servers using `--tsm-servers TSMServer1, TSMServer2` to have two independant backups and this works nicely until you lose a tape when restoring that tape is going to be a nightmare. (read rebuild the whole shadow database)

We started with a copy pool until we filled our tape library up, and then swapped to Protect replication until we found this really did not work very well (really slow and missing files), and IBM surgested we use mmbackup with 2 servers and have two independ backups, which is working very well for us now.

I think if I was going to implement SOBAR I'd want to run mmbackup as well as SOBAR will not give you point in time recovery or partial recovery and is really only a disarster solution. I'd also probably want 3 copies on tape, 1 in SOBAR, and 2x via mmbackup via two backups or via a copy pool

I'm currently thinking to play with HSM and SOBAR on a test system, but have not started yet...... Maybe a talk at the next UG would be helpful on backups, I'm not sure if I want to do one, or if we can find an "expert"

Peter Childs



________________________________________
From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> on behalf of Robert Horton <robert.horton at icr.ac.uk>
Sent: Thursday, March 9, 2023 3:44 PM
To: gpfsug-discuss at gpfsug.org
Subject: [EXTERNAL] [gpfsug-discuss] mmbackup vs SOBAR

CAUTION: This email originated from outside of QMUL. Do not click links or open attachments unless you recognise the sender and know the content is safe.

Hi Folks,

I'm setting up a filesystem for "archive" data which will be aggressively tiered to tape using the Spectrum Protect (or whatever it's called today) Space Management. I would like to have two copies on tape for a) reading back the data on demand b) recovering accidentally deleted files etc c) disaster recovery of the whole filesystem if necessary.

My understanding is:

  1.  Backup and Migration are completely separate things to Spectrum Protect. You can't "restore" from a migrated file nor do a DMAPI read from a backup.
  2.  A SOBAR backup would enable the namespace to be restored if the filesystem were lost but needs all files to be (pre-)migrated and needs the filesystem blocksize etc to match.
  3.  A SOBAR backup isn't much help for restoring individual (deleted) files. There is a dsmmigundelete utility that restores individual stubs but doesn't restore directories etc so you really want a separate backup.

My thinking is to do backups to one (non-replicated) tape pool and migrate to another and run mmimgbackup regularly. I'd then have a path to do a full restore if either set of tapes were lost although it seems rather messy and it's a bit of a pain that SP needs to read everything twice.

So... have I understood that correctly and does anyone have any better / alternative suggestions?

Thanks,
Rob

Robert Horton | Scientific Computing Infrastructure Lead The Institute of Cancer Research | 237 Fulham Road, London, SW3 6JB T +44 (0) 20 7153 5350 | E robert.horton at icr.ac.uk<mailto:robert.horton at icr.ac.uk> | W http://www.icr.ac.uk/<http://www.icr.ac.uk/> | Twitter @ICR_London<https://twitter.com/ICR_London>
Facebook http://www.facebook.com/theinstituteofcancerresearch<http://www.facebook.com/theinstituteofcancerresearch>
Making the discoveries that defeat cancer [ICR Logo]<http://www.icr.ac.uk/>


The Institute of Cancer Research: Royal Cancer Hospital, a charitable Company Limited by Guarantee, Registered in England under Company No. 534147 with its Registered Office at 123 Old Brompton Road, London SW7 3RP.

This e-mail message is confidential and for use by the addressee only. If the message is received by anyone other than the addressee, please return the message to the sender by replying to it and then delete the message from your computer and network.

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org



From pinto at scinet.utoronto.ca  Tue Apr 25 14:27:07 2023
From: pinto at scinet.utoronto.ca (Jaime Pinto)
Date: Tue, 25 Apr 2023 09:27:07 -0400
Subject: [gpfsug-discuss] [EXTERNAL] mmbackup vs SOBAR
In-Reply-To: <LO0P265MB5215BCD77EB158E3456030F6D3649@LO0P265MB5215.GBRP265.PROD.OUTLOOK.COM>
References: <LO4P265MB4050C9CC3B577FBFA7B1CAFDACB59@LO4P265MB4050.GBRP265.PROD.OUTLOOK.COM>
 <DU0PR07MB85133C76E540B16DC4D2D59DA4B59@DU0PR07MB8513.eurprd07.prod.outlook.com>
 <LO0P265MB5215BCD77EB158E3456030F6D3649@LO0P265MB5215.GBRP265.PROD.OUTLOOK.COM>
Message-ID: <dae6da03-c40b-4c13-7446-7a32aef93dd8@scinet.utoronto.ca>

I would be very interested on this session it I could attend it remotely.
Thanks
Jaime

On 4/25/2023 07:55:59, Paul Ward wrote:
> Hi Peter and Robert,
> 
> Sorry for the delayed reply, only occasionally check the mailing list.
> I'm happy to have a MS Teams (or other platform) call about the setup you are talking about, as we've just decommission that kind of environment.
> 
> Spectrum SCALE
> Spectrum Protect with HSM using a dual robot High density tape library with off site copy
> SOBAR - (in theory) implemented.
> 
> Kindest regards,
> Paul
> 
> Paul Ward
> TS Infrastructure Architect
> Natural History Museum
> T: 02079426450
> E: p.ward at nhm.ac.uk
> 
> 
> -----Original Message-----
> From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> On Behalf Of Peter Childs
> Sent: Thursday, March 9, 2023 4:08 PM
> To: gpfsug-discuss at gpfsug.org
> Subject: Re: [gpfsug-discuss] [EXTERNAL] mmbackup vs SOBAR
> 
> I've been told "you really should be using SOBAR" a few times, but never really understood how to do so and the steps involved. I feel sure it should have some kind of white paper, so far I've been thinking to setup some kind of test system, but get a little lost on where to start (and lack of time).
> 
> We currently use mmbackup to x2 servers using `--tsm-servers TSMServer1, TSMServer2` to have two independant backups and this works nicely until you lose a tape when restoring that tape is going to be a nightmare. (read rebuild the whole shadow database)
> 
> We started with a copy pool until we filled our tape library up, and then swapped to Protect replication until we found this really did not work very well (really slow and missing files), and IBM surgested we use mmbackup with 2 servers and have two independ backups, which is working very well for us now.
> 
> I think if I was going to implement SOBAR I'd want to run mmbackup as well as SOBAR will not give you point in time recovery or partial recovery and is really only a disarster solution. I'd also probably want 3 copies on tape, 1 in SOBAR, and 2x via mmbackup via two backups or via a copy pool
> 
> I'm currently thinking to play with HSM and SOBAR on a test system, but have not started yet...... Maybe a talk at the next UG would be helpful on backups, I'm not sure if I want to do one, or if we can find an "expert"
> 
> Peter Childs
> 
> 
> 
> ________________________________________
> From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> on behalf of Robert Horton <robert.horton at icr.ac.uk>
> Sent: Thursday, March 9, 2023 3:44 PM
> To: gpfsug-discuss at gpfsug.org
> Subject: [EXTERNAL] [gpfsug-discuss] mmbackup vs SOBAR
> 
> CAUTION: This email originated from outside of QMUL. Do not click links or open attachments unless you recognise the sender and know the content is safe.
> 
> Hi Folks,
> 
> I'm setting up a filesystem for "archive" data which will be aggressively tiered to tape using the Spectrum Protect (or whatever it's called today) Space Management. I would like to have two copies on tape for a) reading back the data on demand b) recovering accidentally deleted files etc c) disaster recovery of the whole filesystem if necessary.
> 
> My understanding is:
> 
>    1.  Backup and Migration are completely separate things to Spectrum Protect. You can't "restore" from a migrated file nor do a DMAPI read from a backup.
>    2.  A SOBAR backup would enable the namespace to be restored if the filesystem were lost but needs all files to be (pre-)migrated and needs the filesystem blocksize etc to match.
>    3.  A SOBAR backup isn't much help for restoring individual (deleted) files. There is a dsmmigundelete utility that restores individual stubs but doesn't restore directories etc so you really want a separate backup.
> 
> My thinking is to do backups to one (non-replicated) tape pool and migrate to another and run mmimgbackup regularly. I'd then have a path to do a full restore if either set of tapes were lost although it seems rather messy and it's a bit of a pain that SP needs to read everything twice.
> 
> So... have I understood that correctly and does anyone have any better / alternative suggestions?
> 
> Thanks,
> Rob
> 
> Robert Horton | Scientific Computing Infrastructure Lead The Institute of Cancer Research | 237 Fulham Road, London, SW3 6JB T +44 (0) 20 7153 5350 | E robert.horton at icr.ac.uk<mailto:robert.horton at icr.ac.uk> | W http://www.icr.ac.uk/<http://www.icr.ac.uk/> | Twitter @ICR_London<https://twitter.com/ICR_London>
> Facebook http://www.facebook.com/theinstituteofcancerresearch<http://www.facebook.com/theinstituteofcancerresearch>
> Making the discoveries that defeat cancer [ICR Logo]<http://www.icr.ac.uk/>
> 
> 
> The Institute of Cancer Research: Royal Cancer Hospital, a charitable Company Limited by Guarantee, Registered in England under Company No. 534147 with its Registered Office at 123 Old Brompton Road, London SW7 3RP.
> 
> This e-mail message is confidential and for use by the addressee only. If the message is received by anyone other than the addressee, please return the message to the sender by replying to it and then delete the message from your computer and network.
> 
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
> 
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
> 

---
Jaime Pinto - Storage Analyst
SciNet HPC Consortium - www.scinet.utoronto.ca
University of Toronto




From anacreo at gmail.com  Fri Apr 28 11:41:28 2023
From: anacreo at gmail.com (Alec)
Date: Fri, 28 Apr 2023 03:41:28 -0700
Subject: [gpfsug-discuss] [EXTERNAL] mmbackup vs SOBAR
In-Reply-To: <LO0P265MB5215BCD77EB158E3456030F6D3649@LO0P265MB5215.GBRP265.PROD.OUTLOOK.COM>
References: <LO4P265MB4050C9CC3B577FBFA7B1CAFDACB59@LO4P265MB4050.GBRP265.PROD.OUTLOOK.COM>
 <DU0PR07MB85133C76E540B16DC4D2D59DA4B59@DU0PR07MB8513.eurprd07.prod.outlook.com>
 <LO0P265MB5215BCD77EB158E3456030F6D3649@LO0P265MB5215.GBRP265.PROD.OUTLOOK.COM>
Message-ID: <CAGhSTwj1KYguckfzqFNPzHB-FJtkokhTB8z9uy4GM0=Gr2QsKg@mail.gmail.com>

I don't know how in-context this is... but we do our backups using
NetBackup to Data Domain using Accelerated Backups...

My chargeback is through the roof because they think we backup all 600TB of
data in about 10 hours, which is a synthetic backup speed of ~1TB/min.  Our
actual data throughput rate on a single 10G channel is about 1.7GB/s, our
backup reduction on the Data Domain is about 168:1.  Which is phenomenal
since our primary source database is about 18TB, our stats against that DB
is about 600TB of data and our 35 days of backups on the Data Domain takes
about 17TB of space.  Our data is Statistical files and we have a mix of gz
and non-gz files and the Data Domain seems to dedupe and compress either
format fine to the original uncompressed bytes, eg. we gzipped about 30TB
overnight and our backup footprint didn't grow at all... (gotta love
mmfind...)

Our backup performance and our primary disk performance using Spectrum
Scale is pretty much just wire speed and because we can scan millions of
files a minute, and we're able to GZip compress our old data automatically
using a simple GPFS policy, we just haven't bothered creating an "archive"
solution which can create its own headaches, essentially we archive our
data in-place.

Honestly this configuration only has a few weak points... Since we have a
single file system we have to manually break the backups up over multiple
policies by Fileset to achieve an optimal amount of parallelism.  A full
data recovery could take more time than desired.  We replicate our backups
and our primary disk, but if we failover using our disk image replication
we must do a full backup from scratch, this can take 2 weeks to achieve a
proper steady state again.  We could do something to provide relief such as
periodically backing up from the BCP so the backups don't fall off (but
we'd have to pause replication during this).  Or we could have used
GPFS's native data replication and have active/active sites and just backup
from both sites.

My favorite part of the solution... it uses a commodity backup which is
completely off disk... if I needed to restore to any other location I
could, and I am able to rely on the backup team to fully support the
solution without any unique costs or configuration (except the multiple
policies)... yet we're the fastest and largest backup client in the
configuration by miles on a "commodity" solution.

Anyhow, just my two cents about how we've achieved success with the
wonderful Spectrum Scale FS and our standard NetBackup / Data Domain backup
environment.

Please reach out to me privately if anyone wants more details about this
configuration, I'll share what is permissible..

Alec



On Tue, Apr 25, 2023 at 4:58?AM Paul Ward <p.ward at nhm.ac.uk> wrote:

> Hi Peter and Robert,
>
> Sorry for the delayed reply, only occasionally check the mailing list.
> I'm happy to have a MS Teams (or other platform) call about the setup you
> are talking about, as we've just decommission that kind of environment.
>
> Spectrum SCALE
> Spectrum Protect with HSM using a dual robot High density tape library
> with off site copy
> SOBAR - (in theory) implemented.
>
> Kindest regards,
> Paul
>
> Paul Ward
> TS Infrastructure Architect
> Natural History Museum
> T: 02079426450
> E: p.ward at nhm.ac.uk
>
>
> -----Original Message-----
> From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> On Behalf Of
> Peter Childs
> Sent: Thursday, March 9, 2023 4:08 PM
> To: gpfsug-discuss at gpfsug.org
> Subject: Re: [gpfsug-discuss] [EXTERNAL] mmbackup vs SOBAR
>
> I've been told "you really should be using SOBAR" a few times, but never
> really understood how to do so and the steps involved. I feel sure it
> should have some kind of white paper, so far I've been thinking to setup
> some kind of test system, but get a little lost on where to start (and lack
> of time).
>
> We currently use mmbackup to x2 servers using `--tsm-servers TSMServer1,
> TSMServer2` to have two independant backups and this works nicely until you
> lose a tape when restoring that tape is going to be a nightmare. (read
> rebuild the whole shadow database)
>
> We started with a copy pool until we filled our tape library up, and then
> swapped to Protect replication until we found this really did not work very
> well (really slow and missing files), and IBM surgested we use mmbackup
> with 2 servers and have two independ backups, which is working very well
> for us now.
>
> I think if I was going to implement SOBAR I'd want to run mmbackup as well
> as SOBAR will not give you point in time recovery or partial recovery and
> is really only a disarster solution. I'd also probably want 3 copies on
> tape, 1 in SOBAR, and 2x via mmbackup via two backups or via a copy pool
>
> I'm currently thinking to play with HSM and SOBAR on a test system, but
> have not started yet...... Maybe a talk at the next UG would be helpful on
> backups, I'm not sure if I want to do one, or if we can find an "expert"
>
> Peter Childs
>
>
>
> ________________________________________
> From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> on behalf of
> Robert Horton <robert.horton at icr.ac.uk>
> Sent: Thursday, March 9, 2023 3:44 PM
> To: gpfsug-discuss at gpfsug.org
> Subject: [EXTERNAL] [gpfsug-discuss] mmbackup vs SOBAR
>
> CAUTION: This email originated from outside of QMUL. Do not click links or
> open attachments unless you recognise the sender and know the content is
> safe.
>
> Hi Folks,
>
> I'm setting up a filesystem for "archive" data which will be aggressively
> tiered to tape using the Spectrum Protect (or whatever it's called today)
> Space Management. I would like to have two copies on tape for a) reading
> back the data on demand b) recovering accidentally deleted files etc c)
> disaster recovery of the whole filesystem if necessary.
>
> My understanding is:
>
>   1.  Backup and Migration are completely separate things to Spectrum
> Protect. You can't "restore" from a migrated file nor do a DMAPI read from
> a backup.
>   2.  A SOBAR backup would enable the namespace to be restored if the
> filesystem were lost but needs all files to be (pre-)migrated and needs the
> filesystem blocksize etc to match.
>   3.  A SOBAR backup isn't much help for restoring individual (deleted)
> files. There is a dsmmigundelete utility that restores individual stubs but
> doesn't restore directories etc so you really want a separate backup.
>
> My thinking is to do backups to one (non-replicated) tape pool and migrate
> to another and run mmimgbackup regularly. I'd then have a path to do a full
> restore if either set of tapes were lost although it seems rather messy and
> it's a bit of a pain that SP needs to read everything twice.
>
> So... have I understood that correctly and does anyone have any better /
> alternative suggestions?
>
> Thanks,
> Rob
>
> Robert Horton | Scientific Computing Infrastructure Lead The Institute of
> Cancer Research | 237 Fulham Road, London, SW3 6JB T +44 (0) 20 7153 5350 |
> E robert.horton at icr.ac.uk<mailto:robert.horton at icr.ac.uk> | W
> http://www.icr.ac.uk/<http://www.icr.ac.uk/> | Twitter @ICR_London<
> https://twitter.com/ICR_London>
> Facebook http://www.facebook.com/theinstituteofcancerresearch<
> http://www.facebook.com/theinstituteofcancerresearch>
> Making the discoveries that defeat cancer [ICR Logo]<http://www.icr.ac.uk/
> >
>
>
> The Institute of Cancer Research: Royal Cancer Hospital, a charitable
> Company Limited by Guarantee, Registered in England under Company No.
> 534147 with its Registered Office at 123 Old Brompton Road, London SW7 3RP.
>
> This e-mail message is confidential and for use by the addressee only. If
> the message is received by anyone other than the addressee, please return
> the message to the sender by replying to it and then delete the message
> from your computer and network.
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20230428/d9275fef/attachment.htm>

From p.ward at nhm.ac.uk  Fri Apr 28 12:04:18 2023
From: p.ward at nhm.ac.uk (Paul Ward)
Date: Fri, 28 Apr 2023 11:04:18 +0000
Subject: [gpfsug-discuss] [EXTERNAL] mmbackup vs SOBAR
In-Reply-To: <dae6da03-c40b-4c13-7446-7a32aef93dd8@scinet.utoronto.ca>
References: <LO4P265MB4050C9CC3B577FBFA7B1CAFDACB59@LO4P265MB4050.GBRP265.PROD.OUTLOOK.COM>
 <DU0PR07MB85133C76E540B16DC4D2D59DA4B59@DU0PR07MB8513.eurprd07.prod.outlook.com>
 <LO0P265MB5215BCD77EB158E3456030F6D3649@LO0P265MB5215.GBRP265.PROD.OUTLOOK.COM>
 <dae6da03-c40b-4c13-7446-7a32aef93dd8@scinet.utoronto.ca>
Message-ID: <LO0P265MB52154ECEA41DCB69FC1F6BF1D36B9@LO0P265MB5215.GBRP265.PROD.OUTLOOK.COM>

I am happy to Jamie.

I will contact Peter and Robert directly to see if they are interested.

Kindest regards,
Paul

Paul Ward
TS Infrastructure Architect
Natural History Museum
T: 02079426450
E: p.ward at nhm.ac.uk


-----Original Message-----
From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> On Behalf Of Jaime Pinto
Sent: Tuesday, April 25, 2023 2:27 PM
To: gpfsug-discuss at gpfsug.org
Subject: Re: [gpfsug-discuss] [EXTERNAL] mmbackup vs SOBAR

I would be very interested on this session it I could attend it remotely.
Thanks
Jaime

On 4/25/2023 07:55:59, Paul Ward wrote:
> Hi Peter and Robert,
>
> Sorry for the delayed reply, only occasionally check the mailing list.
> I'm happy to have a MS Teams (or other platform) call about the setup you are talking about, as we've just decommission that kind of environment.
>
> Spectrum SCALE
> Spectrum Protect with HSM using a dual robot High density tape library
> with off site copy SOBAR - (in theory) implemented.
>
> Kindest regards,
> Paul
>
> Paul Ward
> TS Infrastructure Architect
> Natural History Museum
> T: 02079426450
> E: p.ward at nhm.ac.uk
>
>
> -----Original Message-----
> From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> On Behalf Of
> Peter Childs
> Sent: Thursday, March 9, 2023 4:08 PM
> To: gpfsug-discuss at gpfsug.org
> Subject: Re: [gpfsug-discuss] [EXTERNAL] mmbackup vs SOBAR
>
> I've been told "you really should be using SOBAR" a few times, but never really understood how to do so and the steps involved. I feel sure it should have some kind of white paper, so far I've been thinking to setup some kind of test system, but get a little lost on where to start (and lack of time).
>
> We currently use mmbackup to x2 servers using `--tsm-servers
> TSMServer1, TSMServer2` to have two independant backups and this works
> nicely until you lose a tape when restoring that tape is going to be a
> nightmare. (read rebuild the whole shadow database)
>
> We started with a copy pool until we filled our tape library up, and then swapped to Protect replication until we found this really did not work very well (really slow and missing files), and IBM surgested we use mmbackup with 2 servers and have two independ backups, which is working very well for us now.
>
> I think if I was going to implement SOBAR I'd want to run mmbackup as
> well as SOBAR will not give you point in time recovery or partial
> recovery and is really only a disarster solution. I'd also probably
> want 3 copies on tape, 1 in SOBAR, and 2x via mmbackup via two backups
> or via a copy pool
>
> I'm currently thinking to play with HSM and SOBAR on a test system, but have not started yet...... Maybe a talk at the next UG would be helpful on backups, I'm not sure if I want to do one, or if we can find an "expert"
>
> Peter Childs
>
>
>
> ________________________________________
> From: gpfsug-discuss <gpfsug-discuss-bounces at gpfsug.org> on behalf of
> Robert Horton <robert.horton at icr.ac.uk>
> Sent: Thursday, March 9, 2023 3:44 PM
> To: gpfsug-discuss at gpfsug.org
> Subject: [EXTERNAL] [gpfsug-discuss] mmbackup vs SOBAR
>
> CAUTION: This email originated from outside of QMUL. Do not click links or open attachments unless you recognise the sender and know the content is safe.
>
> Hi Folks,
>
> I'm setting up a filesystem for "archive" data which will be aggressively tiered to tape using the Spectrum Protect (or whatever it's called today) Space Management. I would like to have two copies on tape for a) reading back the data on demand b) recovering accidentally deleted files etc c) disaster recovery of the whole filesystem if necessary.
>
> My understanding is:
>
>    1.  Backup and Migration are completely separate things to Spectrum Protect. You can't "restore" from a migrated file nor do a DMAPI read from a backup.
>    2.  A SOBAR backup would enable the namespace to be restored if the filesystem were lost but needs all files to be (pre-)migrated and needs the filesystem blocksize etc to match.
>    3.  A SOBAR backup isn't much help for restoring individual (deleted) files. There is a dsmmigundelete utility that restores individual stubs but doesn't restore directories etc so you really want a separate backup.
>
> My thinking is to do backups to one (non-replicated) tape pool and migrate to another and run mmimgbackup regularly. I'd then have a path to do a full restore if either set of tapes were lost although it seems rather messy and it's a bit of a pain that SP needs to read everything twice.
>
> So... have I understood that correctly and does anyone have any better / alternative suggestions?
>
> Thanks,
> Rob
>
> Robert Horton | Scientific Computing Infrastructure Lead The Institute
> of Cancer Research | 237 Fulham Road, London, SW3 6JB T +44 (0) 20
> 7153 5350 | E robert.horton at icr.ac.uk<mailto:robert.horton at icr.ac.uk>
> | W
> http://www.i/
> cr.ac.uk%2F&data=05%7C01%7Cp.ward%40nhm.ac.uk%7Cf850a1c2f0744aaa1c9608
> db459138bb%7C73a29c014e78437fa0d4c8553e1960c1%7C1%7C0%7C63818026224535
> 6209%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT
> iI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=pW%2FX3kf8c53WXrfscjVf
> ejhsKiXZ3y4GiXIN1AG2%2BNg%3D&reserved=0<https://eur03.safelinks.protec/
> tion.outlook.com/?url=http%3A%2F%2Fwww.icr.ac.uk%2F&data=05%7C01%7Cp.w
> ard%40nhm.ac.uk%7Cf850a1c2f0744aaa1c9608db459138bb%7C73a29c014e78437fa
> 0d4c8553e1960c1%7C1%7C0%7C638180262245356209%7CUnknown%7CTWFpbGZsb3d8e
> yJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C30
> 00%7C%7C%7C&sdata=pW%2FX3kf8c53WXrfscjVfejhsKiXZ3y4GiXIN1AG2%2BNg%3D&r
> eserved=0> | Twitter
> @ICR_London<https://eur03.safelinks.protection.outlook.com/?url=https%25
> 3A%2F%2Ftwitter.com%2FICR_London&data=05%7C01%7Cp.ward%40nhm.ac.uk%7Cf
> 850a1c2f0744aaa1c9608db459138bb%7C73a29c014e78437fa0d4c8553e1960c1%7C1
> %7C0%7C638180262245356209%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAi
> LCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=T
> dEByp%2BDrWfX%2F%2Bue7ArI7ice95h8w%2F8QycMlfqbaktk%3D&reserved=0>
> Facebook
> http://www.f/
> acebook.com%2Ftheinstituteofcancerresearch&data=05%7C01%7Cp.ward%40nhm
> .ac.uk%7Cf850a1c2f0744aaa1c9608db459138bb%7C73a29c014e78437fa0d4c8553e
> 1960c1%7C1%7C0%7C638180262245356209%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC
> 4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%
> 7C&sdata=5oAewAF9x2%2BnPVV1iRQUO71UYs37MfqE3jAaEFIHqxg%3D&reserved=0<h
> ttps://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.fa
> cebook.com%2Ftheinstituteofcancerresearch&data=05%7C01%7Cp.ward%40nhm.
> ac.uk%7Cf850a1c2f0744aaa1c9608db459138bb%7C73a29c014e78437fa0d4c8553e1
> 960c1%7C1%7C0%7C638180262245356209%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4
> wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7
> C&sdata=5oAewAF9x2%2BnPVV1iRQUO71UYs37MfqE3jAaEFIHqxg%3D&reserved=0>
> Making the discoveries that defeat cancer [ICR
> Logo]<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%252
> Fwww.icr.ac.uk%2F&data=05%7C01%7Cp.ward%40nhm.ac.uk%7Cf850a1c2f0744aaa
> 1c9608db459138bb%7C73a29c014e78437fa0d4c8553e1960c1%7C1%7C0%7C63818026
> 2245356209%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI
> iLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=pW%2FX3kf8c53WXr
> fscjVfejhsKiXZ3y4GiXIN1AG2%2BNg%3D&reserved=0>
>
>
> The Institute of Cancer Research: Royal Cancer Hospital, a charitable Company Limited by Guarantee, Registered in England under Company No. 534147 with its Registered Office at 123 Old Brompton Road, London SW7 3RP.
>
> This e-mail message is confidential and for use by the addressee only. If the message is received by anyone other than the addressee, please return the message to the sender by replying to it and then delete the message from your computer and network.
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsu/
> g.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss_gpfsug.org&data=05%7C01%7C
> p.ward%40nhm.ac.uk%7Cf850a1c2f0744aaa1c9608db459138bb%7C73a29c014e7843
> 7fa0d4c8553e1960c1%7C1%7C0%7C638180262245356209%7CUnknown%7CTWFpbGZsb3
> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7
> C3000%7C%7C%7C&sdata=vTZc4aUe5tuCKplF1dowgTrAIW4iG2dJIw%2B7WiWJ42c%3D&
> reserved=0
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsu/
> g.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss_gpfsug.org&data=05%7C01%7C
> p.ward%40nhm.ac.uk%7Cf850a1c2f0744aaa1c9608db459138bb%7C73a29c014e7843
> 7fa0d4c8553e1960c1%7C1%7C0%7C638180262245356209%7CUnknown%7CTWFpbGZsb3
> d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7
> C3000%7C%7C%7C&sdata=vTZc4aUe5tuCKplF1dowgTrAIW4iG2dJIw%2B7WiWJ42c%3D&
> reserved=0
>

---
Jaime Pinto - Storage Analyst
SciNet HPC Consortium - http://www.scinet.utoronto.ca/
University of Toronto


_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org