Configuration Storage Plugins

Storage Plugin Base Class

class AdvConfigMgr.BaseConfigStorageManager

Base class for storage managers, efines an expandable storage subsystem for configs.

Also, the two methods; BaseConfigStorageManager.read() and BaseConfigStorageManager.write() need to be overwritten to read and write the data in the format needed.

if the manager is intended to be a ‘standard’ one, in other words, if it will be used for automatic read-all/write-all processes, it must be able to run without passing any data or arguments, all configuration must be done during initialization. if it will only be used standalone or on-demand, you can allow any information to be passed.

allow_create = True

Parameters:force (bool) – True if this will set options even if they are locked

config(config_dict)

Parameters:config_dict – a dictionary with storage specific configuration options., this is called after the storage manager is loaded.

force = False

Parameters:overwrite (bool) – True if this will overwrite options that have existing values

force_strings = False

Parameters:standard (bool) – True if this should be used for read_all or write_all ops

lock_after_read = False

Parameters:priority (int) – the priority of this manager, with smallest being run earlier than larger.

overwrite = True

Parameters:lock_after_read (bool) – True if this will lock the option after reading

read(section_name=None, storage_name=None, **kwargs)

Read from storage and save to the system

Parameters:

• section_name (str or list) – A string or list of sections to read from in the config.
• storage_name (str) – A string name of the storage manager, this can be used to override the configured name.
• kwargs – each storage manager may define its own additional args, but must also implement the final kwargs parameter so that if it is called with other arguments, it wont cause an error.

Returns:

the number of sections / options added

The recommended implementation method us to read from your storage method (database, special file, etc) and store the arguments in a dictionary or dictionary of dictionaries. then pass that dict to BaseConfigStorageManager._save_dict(). that method will take care of writing the data, converting it if needed, making sure that it is allowed to write, handling locked sections and options, etc...

if the implementation tries to pass data directly to the file manager for importing, it will save the data in BaseConfigStorageManager.data() where you can read it, so you should check this before processing.

You shoudl keep track of the number of sections and options written/read and return these at the end:

return self.last_section_count, self.last_option_count

standard = True

Parameters:allow_create (bool) – True if this can create options in the system, even if they are not pre-configured.

storage_name = None

Parameters:force_strings (bool) – If True, the system will convert all options to strings before writing to the manager, and from strings when reading from it.

storage_type_name = 'Base'

Parameters:storage_name (str) – The internal name of the storage manager, must be unique

write(section_name=None, storage_name=None, **kwargs)

Write data from the system and save to your storage

Parameters:

• section_name (str or list) – A string or list of sections to write to.
• storage_name (str) – A string name of the storage manager, this can be used to override the configured name.
• kwargs – each storage manager may define its own additional args, but must also implement the final kwargs parameter so that if it is called with other arguments, it wont cause an error.

Returns:

the number of sections / options written

The recommended implementation method is to call BaseConfigStorageManager._get_dict() which will return a dictionary of the options or dictionary of sections (which are dicts of options) to be saved. You can then iterate through these and save them in your storage system.

if you want to return data direct from the write method, you should copy it to BaseConfigStorageManager.data() after processing.

you shoudl keep track of the number of sections and options written/read and return these at the end:

return self.last_section_count, self.last_option_count

CLI Plugin Class

class AdvConfigMgr.ConfigCLIStorage

Read configuration from the CLI

force = True

True if this will set options even if they are locked

force_strings = False

True if the storage only accepts strings

lock_after_read = True

True if this will lock the option after reading

overwrite = True

True if this will overwrite options that have existing values

read(section_name=None, storage_name='cli', **kwargs)

will take a dictionary and save it to the system :param dict_in: :param storage_name: :return:

reset_cache()

Reloades the cli_parser from the config.

standard = True

True if this should be used for read_all/write_all ops

write(section_name=None, storage_name='cli', **kwargs)

cli does not accept writing options – disabled

Dictionary Plugin Class

class AdvConfigMgr.ConfigSimpleDictStorage

Read configuration from a dictionary.

Keys are section names, values are dictionaries with keys and values that should be present in the section.

read(section_name=None, storage_name='dict', **kwargs)

will take a dictionary and save it to the system :param dict_in: :param storage_name: :return:

standard = False

True if this should be used for read_all/write_all ops

write(section_name=None, storage_name='dict', **kwargs)

will return a dictionary from the system :param storage_name: :return:

File Plugin Class

class AdvConfigMgr.ConfigFileStorage

A file manager that stores config files in a text file

this manager can handle multiple files, as well as a string or list of data, as long as the data is in the format of an ini file. it can also handle scanning a directory or list of directories for all files matching a filter pattern.

if multiple files or filenames are passed, the files read will be processed in the order they are listed, with sections being merged and options overwriting older ones.

if a directory path is passed, the files will be sorted based on the “read_path_order” option and processed in that order.

Parameters:

• delimiters (tuple) – the delimiter between the key and the value
• comment_prefixes (tuple) – this is a tuple of characters that if they occur as the first non-whitespace character of a line, the line is a comment
• inline_comment_prefixes (tuple) – this is a tuple of characters that if they occur elsewhere in the line after a whitespace char, the rest of the line is a comment.
• space_around_delimiters (bool) – True if space should be added around the delimeters.
• strict (bool) – if False, duplicate sections will be merged, if True, duplicate sections will raise an error
• read_filenames (str or list) –

  a filename or list of file names, assumed to be in the current directory if not otherwise specified for reading. These can also be path/globs and the system will attempt to read all files matching that glob filter. for example, the following are all exampels of valid parameters:

  'myfile.ini'
  'dir/myfile.ini'
  'dir/\*.ini'
  ['myfile.ini', 'myotherfile.ini', 'backup_files/myfile_??.ini']
  

  The filename to read from can also be passed during the read operation.

• read_path_order – ‘alpha’ (default) or ‘date’, the order files will be processed if a path is passed.
• filename (str) – If used, the single file to read and write to. (cannot be used with read_filenames write_filename.)
• write_filename (str or None) – the filename to write files to. if None and read_filenames is passed, this will take the first name in the list. if None and read_paths is passed, AND if there is ONLY ONE file in the path that matches the filter, this will use that file. the filename to write to can also be passed during the write operation.
• leave_open (bool) – if True, the file objects will be left open while the config manager is loaded. this can speed up file access, but it also uses up file handles, buffers, memory, and has the possibility of corrupted files.
• create_files (bool) – if False, will not create any files it does not find.
• fail_if_no_file (bool) – if False, will fail and raise an error if the specified filename is not found.
• make_backup_before_writing (bool) – if True, the system will make a backup file before writing the configuration.
• backup_filename (str) – the filename of the backup file. this can have the following formatting keys: ‘{NUM}’ for an incremental number (uses the next available number) ‘{DATE}’ for a date string (‘YYYYMMDD’) ‘{STIME}’ for a 1 second resolution time string (‘HHMMSS’) ‘{MTIME}’ for a 1 minute resolution time string {‘HHMM’) ‘{NAME}’ for the old config file name (without extension)
• backup_path (str) – if not None (the default) this allows the backup file to be in a different location.
• max_backup_number (int) – the max number (assuming a backup file and NUM in the filename)
• encoding (str) –

Returns:

config(config_dict)

Parameters:config_dict (dict) – a dictionary with storage specific configuration options., This is called after the storage manager is loaded.

read(section_name=None, storage_name='file', files=None, encoding=None, **kwargs)

will read an ini file and save it to the system

Parameters:

• section_name (str or list) –
• storage_name (str) –
• file (str or FileObject) –
• encoding (str) –

Returns:

Return type:

int

storage_name = 'file'

the internal name of the storage manager, must be unique

write(section_name=None, storage_name='file', file=None, encoding=None, **kwargs)

will write to an INI file.

MongoDB Plugin Class

Todo

This