How can I discover / access various SE (storage elements) in GlideinWMS environment?

Question

I am submitting jobs to Open Science Grid from a glidein-wms enabled submit hosts such as osg-xsede. I'd like to stage some data on SE nearest to osg-xsede submit host, then distribute data to various SEs available at each destination clusters (more or less as a cache), and access the data on each SE from the worker nodes during the job submission.

How can I distribute data to various SE where osg-xsede will be submitting jobs to? How can make sure that my jobs will be submitted to a cluster with SEs that has my data? Finally, is there a good way to discover SE endpoints from various worker nodes? Is there any documentation written on this subject?

Thanks! Soichi

Answer 1

The simple answer is: There is no uniform and production method to send data to many storage elements. There isn't even a uniform method for requesting access to storage elements.

There are a few things to remember:

• Not all CE's have corresponding SE's.
• Not all SE's have corresponding CE's (though, most do).
• Each SE is managed by a different group of people, with their own authentication methods. Just because you have access to their computing, does not mean you have access to their storage.

The only successful consumer of opportunistic storage on the OSG that I know of is DZero. They went out and asked a small set of storage elements to host their data, and they wrote a custom framework to do what you are asking: maintain the cache, data discovery...

In summary: All the tools are available, SRM, GridFTP, GlideinWMS advertising. It's a people problem. You need to request storage space on all the SE's you want to use.

You may be better off doing what SCEC and SBGrid did, cozy up with 1-2 storage elements, and only use them. Storage elements have a lot of bandwidth available to other compute elements, I wouldn't worry too much about storage being 'close' to the computing. CMS and Atlas already discovered that's not as important as once thought (see Xrootd projects, AAA and FAX).