Help on module bulk_configure:

NAME
    bulk_configure - Tibit Bulk Configuration Utility.

DESCRIPTION
    The Tibit Bulk Configuration Utility is a Python script that parses a CSV file
    which contains provisioning data for one or more OLT or ONU devices. The script
    creates or updates entries in MongoDB for each device row in the CSV file.
    
    
    ===============
    CSV File Format
    ===============
    
    The CSV file contains a header row followed by one or more rows of device
    configuration data, where each row of the file represents configuration for a
    single device. Each column in the header row represents a database atttribute
    and is represented by the format: "[collection][field1][field2]".
    
    For example, the following header column identifies the Address field in the OLT
    Configuration collection in the database (OLT-CFG).
    
    Header Column: [OLT-CFG][OLT][Address]
    
    MongoDB Document:
    
        OLT-CFG = {
            "OLT" : {
                "Address" : "Petaluma, CA",
            }
        }
    
    
    ------------------
    Device Identifiers
    ------------------
    
    The first column must contain the Device ID. The Device ID is used as the key to
    identify a specific document in MongoDB, and maps to the _id field in a MongoDB
    document.
    
    First Header Column: [OLT-CFG][_id]
    
    
    ------
    Arrays
    ------
    
    Arrays (or lists) are represented in a header column by an integer enclosed in
    brackets, where the integer value reprents the index in to the array (e.g.,
    array[0]).
    
    For example, the following header columns reference firmware bank '0' and
    firmware bank '1' in an OLT Configuration document in the database (OLT-CFG).
    
    Header Column #1: [OLT-CFG][OLT][FW Bank Files][0]
    
    Header Column #2: [OLT-CFG][OLT][FW Bank Files][1]
    
    MongoDB Document:
    
        OLT-CFG = {
            "OLT" : {
                "FW Bank Files" : [
                    "R2.0.0-OLT-FW.bin",
                    "R1.3.1-OLT-FW.bin"
                ],
            }
        }
    
    
    ----------------
    Attribute Values
    ----------------
    
    All attribute values are represented as strings in the CSV file. However, values
    must be written into the database with the proper JSON datatype. Since the CSV
    file does not explicitly define a datatype for attributes, the script follows
    specific set of conversion rules when writing values to the database.
    
    The script processes CSV attribute values according to the following ordered
    rules:
    
    1) Values enclosed by triple quotes """<value>""" are always considered a string
       regardless of the format of <value>. Triple quoting is the method to add
       quotes around a value in CSV.
    
    Values with no quotes (or enclosed by a single set of quotes) are processed
    according to the following order.
    
    2) If the value contains a single decimal point and conversion to a Python
       float() succeeds, the value is inserted into the document as a float.
    
    3) If the value conversion to a Python int() succeeds, the value is inserted
       into the document as a integer. Note if a number must be represented in the
       DB as a string, enclose it in triple quotes. For example, """9600""" will
       insert the value into the database as a string instead of an integer.
    
    4) If the value converted to lower case is equal to the string "true" or
       "false", the value is inserted into the document as a Boolean true or false
       value.
    
    5) If all of the above conversions 2..4 fail, the default operation inserts the
       value into the database as a string.
    
    In addition to the conversion rules listed above, the script applies the
    following special cases for blank values:
    * Blank cells (i.e., ",,") are skipped and the document is not updated.
    * Blank strings can be inserted using triple quotes using the following
    format: """""". This is the same as (1) except the <value> is blank and has
    length 0.
    
    
    ------------
    Script Usage
    ------------
    
    Example - bulk create devices from a CSV file:
      ./bulk_configure.py -f ./olt_bulk_config.csv
    
    Example - bulk create devices from a CSV file, overwriting existing entries if they are already in the database:
      ./bulk_configure.py -f ./olt_bulk_config.csv --overwrite
    
    Example - bulk delete devices listed in a CSV file:
      ./bulk_configure.py -f ./olt_bulk_config.csv --remove
    
    
    usage: bulk_configure.py [-h] [-d DB_URI] -f FILE [--insert-missing-fields]
                            [--overwrite] [-r] [-v]
    
    optional arguments:
      -h, --help            show this help message and exit
      -d DB_URI, --db DB_URI
                            Database uri (e.g.,
                            mongodb://127.0.0.1:27017/tibit_pon_controller)
                            (default:
                            mongodb://127.0.0.1:27017/tibit_pon_controller)
      -f FILE, --file FILE  A CSV formatted file containing the device
                            provisioning. (default: None)
      --insert-missing-fields
                            Force insert fields from the CSV which are are
                            missing from the default document. (default: False)
      --overwrite           Force overwrite the entry if it already exists in the
                            database. (default: False)
      -r, --remove          Remove the device entries from the database. (default:
                            False)
      -v, --verbose         Display verbose output. (default: False)

CLASSES
    builtins.dict(builtins.object)
        MongoDbDoc
    builtins.object
        MongoDbDriver
        term_colors
    
    class MongoDbDoc(builtins.dict)
     |  MongoDB Document
     |  
     |  The MongoDB Document class is a wrapper around a dictionary representing
     |  a MongoDB json document. This wrapper class provides "deep" read and write
     |  methods for getting and setting values in a nested dictionary. It also
     |  provides utility methods for automatically converting string values into
     |  the appropriate json datatypes (i.e., int, float, boolean, etc.).
     |  
     |  Method resolution order:
     |      MongoDbDoc
     |      builtins.dict
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, dict_)
     |      MongoDb Document constructor.
     |      
     |      Args:
     |          dict_: A dictionary containing the MongoDB document.
     |  
     |  dumps(self, sort_keys=True, indent=4, separators=(',', ': ')) -> str
     |      Pretty print the document to a string.
     |  
     |  read(self, path:str, default:Union[Any, NoneType]=None) -> Any
     |      Read the value of an attribute in a Mongo document. This method performs
     |      a "deep get" from a nested dictionary. If the attribute does not exist,
     |      the value of 'default' is returned.
     |      
     |      Args:
     |          path: The path to the nested attribute in the document.
     |          default: Default value to return if the attribute does not exist.
     |      
     |      Returns:
     |          dict_: The attribute value.
     |  
     |  write(self, path:str, val:Any, force:bool) -> bool
     |      Write the value of an attribute in a Mongo document. This method performs
     |      a "deep set" in a nested dictionary. If the attribute exists in the document,
     |      the existing value is overwritten by 'val'. If the attribute does not exist,
     |      the write operation depends on the value of 'force'. If the attribute does not
     |      exist and 'force' is true, the new attribute identified by 'path' and
     |      associated value are created in the document. Otherwise, if 'force' is false,
     |      the write operation fails and the document remains unchanged.
     |      
     |      Args:
     |          path: The path to the nested attribute in the document.
     |          val: Value to write in the document for the attribute.
     |          force: Add the field to the document if it does not exist.
     |      
     |      Returns:
     |          result: True if the write operation succeeds, False otherwise.
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors defined here:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from builtins.dict:
     |  
     |  __contains__(self, key, /)
     |      True if D has a key k, else False.
     |  
     |  __delitem__(self, key, /)
     |      Delete self[key].
     |  
     |  __eq__(self, value, /)
     |      Return self==value.
     |  
     |  __ge__(self, value, /)
     |      Return self>=value.
     |  
     |  __getattribute__(self, name, /)
     |      Return getattr(self, name).
     |  
     |  __getitem__(...)
     |      x.__getitem__(y) <==> x[y]
     |  
     |  __gt__(self, value, /)
     |      Return self>value.
     |  
     |  __iter__(self, /)
     |      Implement iter(self).
     |  
     |  __le__(self, value, /)
     |      Return self<=value.
     |  
     |  __len__(self, /)
     |      Return len(self).
     |  
     |  __lt__(self, value, /)
     |      Return self<value.
     |  
     |  __ne__(self, value, /)
     |      Return self!=value.
     |  
     |  __new__(*args, **kwargs) from builtins.type
     |      Create and return a new object.  See help(type) for accurate signature.
     |  
     |  __repr__(self, /)
     |      Return repr(self).
     |  
     |  __setitem__(self, key, value, /)
     |      Set self[key] to value.
     |  
     |  __sizeof__(...)
     |      D.__sizeof__() -> size of D in memory, in bytes
     |  
     |  clear(...)
     |      D.clear() -> None.  Remove all items from D.
     |  
     |  copy(...)
     |      D.copy() -> a shallow copy of D
     |  
     |  fromkeys(iterable, value=None, /) from builtins.type
     |      Returns a new dict with keys from iterable and values equal to value.
     |  
     |  get(...)
     |      D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.
     |  
     |  items(...)
     |      D.items() -> a set-like object providing a view on D's items
     |  
     |  keys(...)
     |      D.keys() -> a set-like object providing a view on D's keys
     |  
     |  pop(...)
     |      D.pop(k[,d]) -> v, remove specified key and return the corresponding value.
     |      If key is not found, d is returned if given, otherwise KeyError is raised
     |  
     |  popitem(...)
     |      D.popitem() -> (k, v), remove and return some (key, value) pair as a
     |      2-tuple; but raise KeyError if D is empty.
     |  
     |  setdefault(...)
     |      D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D
     |  
     |  update(...)
     |      D.update([E, ]**F) -> None.  Update D from dict/iterable E and F.
     |      If E is present and has a .keys() method, then does:  for k in E: D[k] = E[k]
     |      If E is present and lacks a .keys() method, then does:  for k, v in E: D[k] = v
     |      In either case, this is followed by: for k in F:  D[k] = F[k]
     |  
     |  values(...)
     |      D.values() -> an object providing a view on D's values
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes inherited from builtins.dict:
     |  
     |  __hash__ = None
    
    class MongoDbDriver(builtins.object)
     |  MongoDB Driver
     |  
     |  The MongoDB Driver class represents a connection to sepecific database
     |  on MongoDB server. It provides CRUD methods for reading, creating, updating,
     |  and deleting document from the database.
     |  
     |  Methods defined here:
     |  
     |  __init__(self, db_uri='mongodb://127.0.0.1:27017/tibit_pon_controller')
     |      MongoDb Driver Constructor.
     |      
     |      Args:
     |          db_uri: MongoDB URI connection string.
     |  
     |  create(self, db_collection_name, db_doc, test=False)
     |      Create a document in MongodDB. This operation fails if the document already
     |      exists in the database.
     |      
     |      Args:
     |          db_collection_name: The name of the collection in the database.
     |          db_doc: A json-like dictionary containing the document to insert.
     |          test: Boolean to simulate the operation on the database. The database
     |              is not updated during the operation. Used to support the bulk
     |              configure --test option.
     |  
     |  delete(self, db_collection_name, db_key, test=False)
     |      Remove a document from MongodDB.
     |      
     |      Args:
     |          db_collection_name: The name of the collection in the database.
     |          db_key: The key used to identify the document in the database to remove.
     |          test: Boolean to simulate the operation on the database. The database
     |              is not updated during the operation. Used to support the bulk
     |              configure --test option.
     |  
     |  read(self, db_collection_name, db_key)
     |      Retrieve a document from MongodDB.
     |      
     |      Args:
     |          db_collection_name: The name of the collection in the database.
     |          db_key: The key used to identify the document in the database to retrieve.
     |      
     |      Returns:
     |          db_doc: The document identified by db_key or an empty dictionary ({})
     |              if the document does not exist in the database.
     |  
     |  update(self, db_collection_name, db_doc, test=False)
     |      Create or update a document in MongodDB. If already exists in the database,
     |      the existing document is updated. Otherwise, a new document is created in the
     |      database.
     |      
     |      Args:
     |          db_collection_name: The name of the collection in the database.
     |          db_doc: A json-like dictionary containing the document to replace.
     |          test: Boolean to simulate the operation on the database. The database
     |              is not updated during the operation. Used to support the bulk
     |              configure --test option.
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors defined here:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
    
    class term_colors(builtins.object)
     |  # ANSI escape sequences for terminal coloring
     |  
     |  Data descriptors defined here:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
     |  
     |  ----------------------------------------------------------------------
     |  Data and other attributes defined here:
     |  
     |  BOLD = '\x1b[1m'
     |  
     |  ENDC = '\x1b[0m'
     |  
     |  FAIL = '\x1b[91m'
     |  
     |  HEADER = '\x1b[95m'
     |  
     |  OKBLUE = '\x1b[94m'
     |  
     |  OKCYAN = '\x1b[96m'
     |  
     |  OKGREEN = '\x1b[92m'
     |  
     |  UNDERLINE = '\x1b[4m'
     |  
     |  WARNING = '\x1b[93m'

FUNCTIONS
    ERROR(*args, **kwargs)
    
    HIGHLIGHT_ER(condition, *args, **kwargs)
    
    HIGHLIGHT_OK(condition, *args, **kwargs)
    
    HIGHLIGHT_WA(condition, *args, **kwargs)
    
    INFO(*args, **kwargs)
    
    RE_MATCH_ISO8601_TIMESTAMP = match(string=None, pos=0, endpos=9223372036854775807, *, pattern=None) method of _sre.SRE_Pattern instance
        Matches zero or more characters at the beginning of the string.
    
    WARNING(*args, **kwargs)
    
    main(args)
        Main function for BulkConfigure script.
        
        Args:
            args: Command line arguments.
    
    re_db_timestamp_is_valid(str_val)
        Parse and validate a string representing an PON Controller Timestamp.
        
        This function converts the PON Controller Timestamp to ISO8601 format before validating it.
        
        Args:
            str_val: A PON Controller timestamp value.
        
        Returns:
            is_valid: Boolean value of 'True' if the timestamp is valid; 'False' otherwise.
    
    re_parse_bracketed_path(path)
        Parse a string representing the path to an attributed in a MongoDB document.
        
        MongoDB documents are a nested python dictionary. The path identifying an attribute
        in the document uses a braketed format: "[collection][node][array][0]"
        
        Args:
            path: The path identifying an attribute using the bracketed format.
        
        Returns:
            parsed_path: A list of string represting the elements or individual nodes in the path.
    
    re_parse_elem_list(elem)
        Parse the list key and list index from a path element.
        
        Path elements that represent arrays have the following
        format: "<list_key>[<list_index>]". This function parses the key and
        index from a path element.
        
        Args:
            elem: An element from a path string.
        
        Returns:
            list_key: A string identifying the list (or array) attribute name.
            list_index: An integer representing the index into the list.

DATA
    Any = typing.Any
    Optional = typing.Optional
    RE_MATCH_BRACKETED_PATH = re.compile('\\[(.*?)\\]')
    RE_MATCH_LIST_INDEX = re.compile('\\[(\\d+)\\]$')

FILE
    /home/mhartling/repo/tibit/mhartling-pon_manager/bulk_configure/bulk_configure.py


