nems.uri module


Where the filesystem organization of nems directories are decided, and generic methods for saving and loading resources over HTTP or to local files.

nems.uri.log = <Logger nems.uri (INFO)>[source]

REMOVED SVD – to be deleted

class NumpyAwareJSONEncoder(jsonlib.JSONEncoder):

#DEPRECATED. DELETE ME? #For serializing Numpy arrays safely as JSONs. From: # def default(self, obj):

if issubclass(type(obj), Distribution):

return obj.tolist()

if issubclass(type(obj), NemsModule):

return obj.data_dict

if isinstance(obj, np.ndarray): # and obj.ndim == 1:

return obj.tolist()

return jsonlib.JSONEncoder.default(self, obj)


Returns the local filepath if it is a local URI, else None. uri - string


Returns the URL if it is a HTTP/HTTPS URI, else None.


Returns the URI if it is a .tar.gz URI, else None.

nems.uri.save_resource(uri, data=None, json=None)[source]

For saving a resource to a URI. Throws an exception if there was a problem saving.


Loads and returns the resource (probably a JSON) found at URI.


Reads NGINX directory listing at URI and returns a list of URIs that end in .tar.gz that were found in the HTML directory listing.

NOTE: This is a BRITTLE HACK and should not preferred in general to getting a list of files from something smarter, like a database that manages ‘batches’. Ideally, such a database would return a JSON containing a list of URIs.