How to Deal with Configuration and Credential files in Python
An efficient way to handle different connections in a single file!
Published in
2 min readOct 10, 2021
Data Scientists and Data Engineers may need to access many systems at once. For example, in ETL/ELT jobs there is a chance to extract and store data in different sources like Hadoop, BigQuery, MySQL, PostgreSQL, Redshift, Snowflake, and so on. In addition, there can be different users having different credentials and finally, there can be different environments, like test, stage and production. In this post, we will explain how to deal with different credentials in Python using one configuration file.
The configparser Module
The configparser module is built to handle different credentials in one file. Let’s see how to create a configuration file programmatically. Our goal is to create the credentials of different connections.
Create the Configuration File
import configparserconfig = configparser.ConfigParser()# Connection for the user George
config['George'] = {'username': 'george',
'password': 'AStrongPassword01'}
