epsman.repo.pkgFiles module

epsman

Local python script for job packaging.

Can be called from Fabric for remote run case, only requires standard libs.

May be a better way to do this?

15/01/20 Change file type pattern matching from globPat to rePat - fixes bug with some files being ignored erroneously.

01/01/20 v1

epsman.repo.pkgFiles.buildPkg(archName, fileList, pkgDir, archMode='w', cType=14)

Build pkg zip from fileList

Parameters:

• archName (str or Path object) – Archive to write.

• fileList (list of strings or Path objects) – Files to include (full paths)

• pkgDir (str or Path object) – Directory to use as root in archive

• archMode (char, optional, default = 'w') – Set to ‘w’rite or ‘a’ppend to existing archive.

• cType (int, default = zipfile.ZIP_LZMA (=14)) – Compression level.

• TODO –

• case? (- Check if arch exists for 'w') –

• add? (- File size checks to) –

• dirs (- Summary for files &) –

• level. (and verbosity) –

epsman.repo.pkgFiles.checkArch(archName)

Test archive & return info if OK

epsman.repo.pkgFiles.convert_bytes(num)

This function will convert bytes to MB…. GB… etc

epsman.repo.pkgFiles.getFilesPkg(pkgDir, globPat='/**/*', rePat=None, recursive=True)

Glob pkgDir with globPat, and optional re matching with rePat.

Used for getting file lists for packaging ePS job dirs.

Parameters:

• pkgDir (str or Path object) – Directory to search.

• globPat (str, optional, default = r”/**/*”) – Default pattern for globbing, will search dir for all files. Supports basic pattern matching, e.g. r”/**/*[!zip], but note glob matches chars - use re for more control.

• rePat (str, optional, default = None) – Regular expression for filtering glob output. E.g. rePat = “.*substring.*” to search for ‘substring’ in glob output. rePat = “.*substring.*$(?<zip)” to exclude zip files.

• recursive (bool, optional, default = True) – Recursive glob: if True, search subdirs too (with ** pattern).

epsman.repo.pkgFiles.setJobRoot(nbFileName, jobSchema)

Define job dir schema from processed notebook filename.

Parameters:

• nbFileName (str or Path) – Notebook file defining job.

• jobSchema (#) –

• mol/jName_XX-XXeV/ (# - '2016' Jobs defined as) –

• eV (# - '2016sub' Jobs defined by) –

• subdirs (with) –

• mol/*_XX-XXeV/jName (as) –

• mol/jName/ (# - '2019' Jobs defined as) –

• schema (# For 2019) –

• interleaved (energies are) –

• jobs. (while for 2016 schema they are treated independently with different) –

• 25/07/20 (testing new logic here, jobSchema currently bypassed.) –

• jobs (For new style) –

• file (should add checking for job .conf) –

• parsing. (rather than notebook name) –