Utilfuncs API¶

csv module¶

Utility csv functions for reading/writing/appending data.

utilfuncs.csv.append_row(csvpath: Union[pathlib.Path, str], row: List[str], delimiter=',', encoding='utf-8') → None¶

Appends a single row to the csv file specified. The file must already exist.

Parameters

csvpath – path to csv file
row – A list of strings
delimiter – Row delimiter, defaults to “,”
encoding – File encoding, defaults to “utf-8”

utilfuncs.csv.append_rows(csvpath: Union[pathlib.Path, str], list_of_rows: List[List[str]], delimiter=',', encoding='utf-8')¶

Appends multiple rows to the csv file specified. The file must already exist.

Parameters

csvpath – path to csv file
list_of_rows – 2D list of strings
delimiter – Row delimiter, defaults to “,”
encoding – File encoding, defaults to “utf-8”

utilfuncs.csv.get_rows(csvpath: Union[pathlib.Path, str], skip_header=False, delimiter=',', encoding='utf-8') → Generator[List[str], None, None]¶

get_rows yields rows from the csv file specified. The caller must iterate over the returned generator to receive the rows.

Parameters

csvpath – Path to csv file
skip_header – Whether to skip header row, default False
delimiter – Row delimiter character, default “,”
encoding – File encoding, default “utf-8”

Yields

List of strings representing rows

utilfuncs.csv.get_rows_by_column_filter(filepath: Union[pathlib.Path, str], col_index: int, col_value: Any, delimiter=',', encoding='utf-8') → Generator[List[str], None, None]¶

Returns rows where certain values occur in a certain column. Like v-lookups in excel.

Parameters

filepath – Path of file
col_index – 0 based col index on which to filter on
col_value – Value of column to filter on
delimiter – Row delimiter. Defaults to “,”.
encoding – File encoding. Defaults to “utf-8”.

Yields

List of strings representing rows

utilfuncs.csv.write_rows(csvpath: Union[pathlib.Path, str], list_of_rows: List[List[str]], delimiter=',', encoding='utf-8') → None¶

Write rows to csv file.

Parameters

csvpath – Path to csv file
list_of_rows – 2D list of strings
delimiter – Row delimiter character, default “,”
encoding – File encoding, default “utf-8”

file module¶

utilfuncs.file.all_files_under(dirpath: Union[pathlib.Path, str], extension='') → List[pathlib.Path]¶

Iterates through all files that are under the given dir with the given extension ex ‘.log’

Parameters

path – The directory to scan
extension (str, optional) – The extensions to consider if provided (with period)

Returns

List of Path objects or empty list if no files in directory

utilfuncs.file.create_dir_if_not_exists(dirpath: Union[pathlib.Path, str]) → None¶

Creates a directory and all intermediate directories if they don’t exist

Parameters: dirpath – Path to dir

utilfuncs.file.delete_old_files(dirpath: Union[pathlib.Path, str], days: int, extension='') → List[pathlib.Path]¶

Deletes files older than a certain number of days in a given folder and returns a list of files that were deleted. Non recursive.

Parameters

dirpath – Path for directory
days – Age of files in days
extension (str, optional) – File extension including period prefix

Returns

List of files that were deleted

utilfuncs.file.get_base64_encoded_contents(filepath: Union[pathlib.Path, str]) → str¶

Returns a b64 encoded string representing file contents

Parameters: filepath – Path to file
Returns: B64 encoded string

utilfuncs.file.get_contents_as_utf8(filepath: Union[pathlib.Path, str]) → str¶

Returns a string representing contents of the file converted to utf-8.

Parameters: filepath – Path to file
Returns: File content as utf-8 string

utilfuncs.file.get_ext(filepath: Union[pathlib.Path, str]) → str¶

Returns the extension of the file with prepended period (.) .. rubric:: Example

Fetching extension:

>>> get_ext("file.txt")
".txt"

Parameters: filepath – Path of file
Returns: Period prefixed extension for the given file

utilfuncs.file.move_dir(src: Union[pathlib.Path, str], dest: Union[pathlib.Path, str]) → Union[pathlib.Path, str]¶

Moves dir from source to destination and returns new dir path.

Parameters

src – Path to src
dest – Path to new parent dir

Returns

Path to directory under dest

utilfuncs.file.move_file(src_file: Union[pathlib.Path, str], dest_dir: Union[pathlib.Path, str]) → Union[pathlib.Path, str]¶

Moves file to dest dir and returns file path in destination.

Parameters

src_file – Path to src
dest_dir – Path to dir

Returns

Path of file in destination

utilfuncs.file.move_files(src_dir: Union[pathlib.Path, str], dest_dir: Union[pathlib.Path, str], ext='') → None¶

Moves all files of one extension from src to dest

Parameters

src_dir – Source directory containing files to move
dest_dir – Destination directory
ext (str, optional) – Target files to move by extension. Defaults to “”.

utilfuncs.file.move_files_matching_regex(src_dir: Union[pathlib.Path, str], dest_dir: Union[pathlib.Path, str], pattern: str) → None¶

Moves files matching regex to destination, non recursively.

Parameters

src_dir – Path to src dir
dest_dir – Path to dest dir
pattern – Glob pattern

utilfuncs.file.rename_file(filepath: Union[pathlib.Path, str], new_name: str) → Union[pathlib.Path, str]¶

Renames the given file with new name and returns path to renamed file.

Parameters

filepath – Path to existing file
new_name – New name to apply

Returns

Path to renamed file

utilfuncs.file.zipdir(dirpath: Union[pathlib.Path, str], zip_path: Optional[Union[pathlib.Path, str]] = None, delete_dir_afterwards=False, top_level_dir=True) → pathlib.Path¶

Creates a zip archive which contains the directory specified in dirPath.

Parameters

dirpath – Path to directory
zip_path (PathLike, optional) – Path to final zip file including name. If not provided, the zip file is written in dirpath’s parent with the dirpath name and .zip extension.
delete_dir_afterwards (bool, optional) – Whether to delete the orignal dir after compression. Defaults to False.
top_level_dir (bool, optional) – Whether to have dirpath folder inside the zip or to have dirpath files directly inside the zip. Defaults to True.

Returns

Path of compressed zip file

utilfuncs.file.zipfiles(zip_path: Union[pathlib.Path, str], files_or_directories: Sequence[Union[pathlib.Path, str]]) → None¶

Creates a zip archive from list of file or directory paths in destination.

Parameters

zip_path – Path of zip file with proper name and extension
file_paths – List of Paths

misc module¶

utilfuncs.msc.filter_by_glob(collection: Sequence[Any], pattern: str) → Sequence[Any]¶

Returns a filtered collection based on which items match the provided string pattern.

Parameters

collection – Sequence to filter on
pattern – String, optionally containing a wildcard ‘*’ character

Returns

Subset of items in the collection that match the glob pattern

utilfuncs.msc.get_timestamp(date_only=False)¶

Returns the timestamp string for current day :param date_only: True or False :type date_only: bool, optional

Returns: Timestamp (str)

utilfuncs.msc.is_substr(string: str, substr: str, ignorecase=False) → bool¶

Tests whether a string has a given substring. Ignore case parameter for flexibility.

Parameters

string – Full string
substr – string segment
ignorecase (bool, optional) – Whether to ignore case. Defaults to False.

Returns

True or False

utilfuncs.msc.keep_retrying_on_specific_error(error_substr: str, sleep_seconds: int, func: Callable, *args, **kwargs) → Any¶

Calls the provided func with its args and kwargs as long as an exception is raised containing the error_substr. Sleeps for the specified number of seconds between each call. If an error occurs and the error_substr is not part of the error message, it is allowed to raise.

Parameters

error_substr – Specific error string on which to retry
sleep_seconds – Number of seconds to sleep between invocations
func – Function to invoke with args and kwargs. Its return values will be fowarded to the caller.

Returns

Whatever the provided function ought to return.