Python Dependency Injection
Python Dependency Injection
io/blog/python-dependency-injection/
testdriven.io
In this post, we'll show you how to implement Dependency Injection as you
develop an app for plotting historic weather data. After developing the initial app,
using Test-Driven Development, you'll refactor it using Dependency Injection to
decouple pieces of the app to make it easier to test, extend, and maintain.
By the end of this post, you should be able to explain what Dependency Injection
is and implement it in Python with Test-Driven Development (TDD).
3. Most languages that allow for the passing of objects and functions as parameters
support it. You hear more about Dependency Injection in Java and C#, though,
since it's difficult to implement. On the other hand, thanks to Python's dynamic
typing along with its duck typing system, it's easy to implement and thus less
noticeable. Django, Django REST Framework, and FastAPI all utilize
Dependency Injection.
Benefits:
3. Tests doesn't have to change every time that we extend our application
1 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
Scenario:
1. You've decided to build an app for drawing plots from weather history data.
3. Your goal is to draw a plot of that data to see how temperature changed over
time.
First, create (and activate) a virtual environment. Then, install pytest and
Matplotlib:
Since we need to read historic weather data from a CSV file, the read method
should meet the following criteria:
THEN data from CSV should be returned in a dictionary where the keys are
datetime strings in ISO 8601 format ('%Y-%m-%dT%H:%M:%S.%f') and the
values are temperatures measured at that moment
import datetime
from pathlib import Path
2 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
BASE_DIR = Path(__file__).resolve(strict=True).parent
def test_read():
app = App()
for key, value in
app.read(file_name=Path(BASE_DIR).joinpath('london.csv')).items():
assert datetime.datetime.fromisoformat(key)
assert value - 0 == value
1. every key is an ISO 8601 formatted date time string (using the fromisoformat
function from datetime package)
The fromisoformat method from the datetime package was added in Python
3.7. Refer to the official Python docs for more info.
Now to implement the read method, to make the test pass, add new file called
app.py:
import csv
import datetime
from pathlib import Path
BASE_DIR = Path(__file__).resolve(strict=True).parent
class App:
3 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
as file:
reader = csv.reader(file)
next(reader) # Skip header row.
for row in reader:
hour = datetime.datetime.strptime(row[0],
'%d/%m/%Y %H:%M').isoformat()
temperature = float(row[2])
temperatures_by_hour[hour] = temperature
return temperatures_by_hour
Here, we added an App class with a read method that takes a file name as a
parameter. After opening and reading the contents of the CSV, the appropriate
keys (date) and values (temperature) are added to a dictionary which is
eventually returned.
Assuming that you've downloaded the weather data as london.csv, the test
should now pass:
test_app.py .
[100%]
WHEN the draw method is called with a dictionary where the keys are datetime
strings in ISO 8601 format ('%Y-%m-%dT%H:%M:%S.%f') and the values are
4 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
THEN the data should be drawn to a line plot with the time on the X axis and
temperature on the Y axis
def test_draw(monkeypatch):
plot_date_mock = MagicMock()
show_mock = MagicMock()
monkeypatch.setattr(matplotlib.pyplot, 'plot_date',
plot_date_mock)
monkeypatch.setattr(matplotlib.pyplot, 'show',
show_mock)
app = App()
hour = datetime.datetime.now().isoformat()
temperature = 14.52
app.draw({hour: temperature})
_, called_temperatures = plot_date_mock.call_args[0]
assert called_temperatures == [temperature] # check
that plot_date was called with temperatures as second arg
show_mock.assert_called() # check that show is called
import datetime
from pathlib import Path
from unittest.mock import MagicMock
import matplotlib.pyplot
Since we don't want to show the actual plots during the test runs, we used
monkeypatch to mock the plot_date function from matplotlib. Then, the
method under test is called with single temperature. At the end, we checked that
plot_date was called correctly (X and Y axis) and that show was called.
You can read more about monkeypatching with pytest here and more about
mocking here.
5 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
2. It must transform this dictionary into two vectors that can be used in the plot:
dates and temperatures.
dates = matplotlib.dates.date2num(dates)
matplotlib.pyplot.plot_date(dates, temperatures,
linestyle='-')
matplotlib.pyplot.show()
Imports:
import csv
import datetime
from pathlib import Path
import matplotlib.dates
import matplotlib.pyplot
test_app.py ..
6 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
[100%]
app.py:
import csv
import datetime
from pathlib import Path
import matplotlib.dates
import matplotlib.pyplot
BASE_DIR = Path(__file__).resolve(strict=True).parent
class App:
return temperatures_by_hour
7 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
dates.append(datetime.datetime.fromisoformat(date))
temperatures.append(temperature)
dates = matplotlib.dates.date2num(dates)
matplotlib.pyplot.plot_date(dates, temperatures,
linestyle='-')
matplotlib.pyplot.show()
test_app.py:
import datetime
from pathlib import Path
from unittest.mock import MagicMock
import matplotlib.pyplot
BASE_DIR = Path(__file__).resolve(strict=True).parent
def test_read():
app = App()
for key, value in
app.read(file_name=Path(BASE_DIR).joinpath('london.csv')).items():
assert datetime.datetime.fromisoformat(key)
assert value - 0 == value
def test_draw(monkeypatch):
plot_date_mock = MagicMock()
show_mock = MagicMock()
monkeypatch.setattr(matplotlib.pyplot, 'plot_date',
plot_date_mock)
monkeypatch.setattr(matplotlib.pyplot, 'show',
show_mock)
app = App()
hour = datetime.datetime.now().isoformat()
temperature = 14.52
8 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
app.draw({hour: temperature})
_, called_temperatures = plot_date_mock.call_args[0]
assert called_temperatures == [temperature] # check
that plot_date was called with temperatures as second arg
show_mock.assert_called() # check that show is called
You have all that you need to run your application for plotting temperatures by
hour form the selected CSV file.
if __name__ == '__main__':
import sys
file_name = sys.argv[1]
app = App()
temperatures_by_hour = app.read(file_name)
app.draw(temperatures_by_hour)
When app.py runs, it first reads the CSV file from the command line argument
assigned to file_name and then it draws the plot.
9 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
Alright. We finished our initial iteration of our app for plotting historical weather
data. It's working as expected and we're happy to use it. That said, it's tightly
coupled with a CSV. What if you wanted to use a different data format? Like a
JSON payload from an API. This is where Dependency Injection comes into play.
import datetime
from pathlib import Path
BASE_DIR = Path(__file__).resolve(strict=True).parent
def test_read():
app = App()
for key, value in
app.read(file_name=Path(BASE_DIR).joinpath('london.csv')).items():
assert datetime.datetime.fromisoformat(key)
assert value - 0 == value
The test here is the same as our test for test_read in test_app.py.
Second, add a new file called urban_climate_csv.py. Inside that file, create a
class called DataSource with a read method:
import csv
import datetime
from pathlib import Path
10 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
BASE_DIR = Path(__file__).resolve(strict=True).parent
class DataSource:
return temperatures_by_hour
This is same as the read method in our initial app with one difference: We're
using kwargs because we want to have the same interface for all our data
sources. So, we could add new readers as necessary based on the source of the
data.
For example:
csv_reader = DataSource()
reader.read(file_name='foo.csv')
json_reader = DataSource()
reader.read(file_name='foo.json')
api_reader = DataSource()
11 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
reader.read(url='https://foo.bar')
test_app.py ..
[ 66%]
test_urban_climate_csv.py .
[100%]
def test_read():
hour = datetime.datetime.now().isoformat()
temperature = 14.52
temperature_by_hour = {hour: temperature}
data_source = MagicMock()
data_source.read.return_value = temperature_by_hour
app = App(
data_source=data_source
)
assert app.read(file_name='something.csv') ==
temperature_by_hour
12 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
Update the test for draw too. Again, we need to inject the data source to App,
which can be "anything" with an expected interface -- so MagicMock will do:
def test_draw(monkeypatch):
plot_date_mock = MagicMock()
show_mock = MagicMock()
monkeypatch.setattr(matplotlib.pyplot, 'plot_date',
plot_date_mock)
monkeypatch.setattr(matplotlib.pyplot, 'show',
show_mock)
app = App(MagicMock())
hour = datetime.datetime.now().isoformat()
temperature = 14.52
app.draw({hour: temperature})
_, called_temperatures = plot_date_mock.call_args[0]
assert called_temperatures == [temperature] # check
that plot_date was called with temperatures as second arg
show_mock.assert_called() # check that show is called
import datetime
import matplotlib.dates
import matplotlib.pyplot
class App:
13 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
dates.append(datetime.datetime.fromisoformat(date))
temperatures.append(temperature)
dates = matplotlib.dates.date2num(dates)
matplotlib.pyplot.plot_date(dates, temperatures,
linestyle='-')
matplotlib.pyplot.show(block=True)
if __name__ == '__main__':
import sys
from urban_climate_csv import DataSource
file_name = sys.argv[1]
app = App(DataSource())
temperatures_by_hour = app.read(file_name=file_name)
app.draw(temperatures_by_hour)
import datetime
def test_read():
reader = DataSource()
for key, value in
reader.read(file_name='london.csv').items():
assert datetime.datetime.fromisoformat(key)
assert value - 0 == value
14 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
test_app.py ..
[ 66%]
test_urban_climate_csv.py .
[100%]
Now that we've decoupled our App from the data source, we can easily add a
new source.
Let's use data from the OpenWeather API. Go ahead and download a pre-
downloaded response from the API: here. Save it as moscow.json.
Feel free to register with the OpenWeather API and grab historical data for a
different city if you'd prefer.
Add a new file called test_open_weather_json.py, and write a test for a read
method:
import datetime
def test_read():
reader = DataSource()
for key, value in
reader.read(file_name='moscow.json').items():
15 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
assert datetime.datetime.fromisoformat(key)
assert value - 0 == value
Since we're using the same interface to apply Dependency Injection, this test
should look very similar to test_read in test_urban_climate_csv.
In statically-typed languages, like Java and C#, all data sources should
implement the same interface -- i.e., IDataSource. Thanks to duck typing in
Python, we can just implement methods with the same name that takes the same
arguments (**kwargs) for each of our data sources:
import json
import datetime
class DataSource:
return temperatures_by_hour
So, we used the json module to read and load a JSON file. Then, we extracted
the data in a similar manner as we did before. This time we used the
fromtimestamp function because times of measurements are written in Unix
timestamp format.
16 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
if __name__ == '__main__':
import sys
from open_weather_json import DataSource
file_name = sys.argv[1]
app = App(DataSource())
temperatures_by_hour = app.read(file_name=file_name)
app.draw(temperatures_by_hour)
You should see a plot with data from the selected JSON file.
3. Implementing an interface for a new data source is fairly simple as well (you just
need to know the shape of the data)
So, we can now extend the codebase with simple and predictable steps without
having to touch tests that are already written or change the main application.
That's powerful. You could now have a developer focus solely on adding new
data sources without them ever needing to understand or have context on the
main application. That said, if you do need to onboard a new developer who does
need to have context on the entire project, it can take longer for them to get up to
speed due to the decoupling.
Moving right along, let's decouple the plotting portion from the app so we can
more easily add new plotting libraries. Since this will be a similar process to the
data source decoupling, think through the steps on your own before reading the
rest of this section.
def test_draw(monkeypatch):
17 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
plot_date_mock = MagicMock()
show_mock = MagicMock()
monkeypatch.setattr(matplotlib.pyplot, 'plot_date',
plot_date_mock)
monkeypatch.setattr(matplotlib.pyplot, 'show',
show_mock)
app = App(MagicMock())
hour = datetime.datetime.now().isoformat()
temperature = 14.52
app.draw({hour: temperature})
_, called_temperatures = plot_date_mock.call_args[0]
assert called_temperatures == [temperature] # check
that plot_date was called with temperatures as second arg
show_mock.assert_called() # check that show is called
As we can see, it's coupled with Matplotlib. A change to the plotting library will
require a change to the tests. This is something you really want to avoid.
Let's extract the plotting part of our app into its own class much like we did for the
reading in of the data source.
import datetime
from unittest.mock import MagicMock
import matplotlib.pyplot
def test_draw(monkeypatch):
plot_date_mock = MagicMock()
show_mock = MagicMock()
monkeypatch.setattr(matplotlib.pyplot, 'plot_date',
plot_date_mock)
monkeypatch.setattr(matplotlib.pyplot, 'show',
show_mock)
18 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
plot = Plot()
hours = [datetime.datetime.now()]
temperatures = [14.52]
plot.draw(hours, temperatures)
_, called_temperatures = plot_date_mock.call_args[0]
assert called_temperatures == temperatures # check that
plot_date was called with temperatures as second arg
show_mock.assert_called() # check that show is called
import matplotlib.dates
import matplotlib.pyplot
class Plot:
hours = matplotlib.dates.date2num(hours)
matplotlib.pyplot.plot_date(hours, temperatures,
linestyle='-')
matplotlib.pyplot.show(block=True)
This is what our interface will look like for all future Plot classes. So, in this case,
our test will stay the same as long as this interface and the underlying
matplotlib methods stay the same.
19 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
injection
collected 2 items
test_app.py ..
[ 40%]
test_matplotlib_plot.py .
[ 60%]
test_open_weather_json.py .
[ 80%]
test_urban_climate_csv.py .
[100%]
import datetime
from unittest.mock import MagicMock
def test_read():
hour = datetime.datetime.now().isoformat()
temperature = 14.52
temperature_by_hour = {hour: temperature}
data_source = MagicMock()
data_source.read.return_value = temperature_by_hour
app = App(
data_source=data_source,
plot=MagicMock()
)
assert app.read(file_name='something.csv') ==
temperature_by_hour
def test_draw():
20 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
plot_mock = MagicMock()
app = App(
data_source=MagicMock,
plot=plot_mock
)
hour = datetime.datetime.now()
iso_hour = hour.isoformat()
temperature = 14.52
temperature_by_hour = {iso_hour: temperature}
app.draw(temperature_by_hour)
plot_mock.draw.assert_called_with([hour], [temperature])
import datetime
class App:
dates.append(datetime.datetime.fromisoformat(date))
21 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
temperatures.append(temperature)
self.plot.draw(dates, temperatures)
Test:
test_app.py ..
[ 40%]
test_matplotlib_plot.py .
[ 60%]
test_open_weather_json.py .
[ 80%]
test_urban_climate_csv.py .
[100%]
if __name__ == '__main__':
import sys
from open_weather_json import DataSource
from matplotlib_plot import Plot
file_name = sys.argv[1]
app = App(DataSource(), Plot())
22 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
temperatures_by_hour = app.read(file_name=file_name)
app.draw(temperatures_by_hour)
Adding Plotly
import datetime
from unittest.mock import MagicMock
import plotly.graph_objects
def test_draw(monkeypatch):
figure_mock = MagicMock()
monkeypatch.setattr(plotly.graph_objects, 'Figure',
figure_mock)
scatter_mock = MagicMock()
monkeypatch.setattr(plotly.graph_objects, 'Scatter',
scatter_mock)
plot = Plot()
hours = [datetime.datetime.now()]
temperatures = [14.52]
plot.draw(hours, temperatures)
call_kwargs = scatter_mock.call_args[1]
assert call_kwargs['y'] == temperatures # check that
plot_date was called with temperatures as second arg
figure_mock().show.assert_called() # check that show is
called
23 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
It's basically the same as the matplotlib Plot test. The major change is how the
objects and methods from Plotly are mocked.
import plotly.graph_objects
class Plot:
fig = plotly.graph_objects.Figure(
data=[plotly.graph_objects.Scatter(x=hours,
y=temperatures)]
)
fig.show()
test_app.py ..
[ 33%]
test_matplotlib_plot.py .
[ 50%]
test_open_weather_json.py .
[ 66%]
test_plotly_plot.py .
[ 83%]
test_urban_climate_csv.py .
[100%]
24 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
if __name__ == '__main__':
import sys
from open_weather_json import DataSource
from plotly_plot import Plot
file_name = sys.argv[1]
app = App(DataSource(), Plot())
temperatures_by_hour = app.read(file_name=file_name)
app.draw(temperatures_by_hour)
Run your app with moscow.json to see the new plot in your browser:
Adding Configuration
At this point, we can easily add and use different data sources and plotting
libraries in our application. Our tests are no longer coupled with the
implementation. That being said, we still need to make edits to the code to add a
new data source or plotting library:
if __name__ == '__main__':
import sys
from open_weather_json import DataSource
from plotly_plot import Plot
file_name = sys.argv[1]
25 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
Although it's only a short snippet of code, we can take Dependency Injection one
step further and eliminate the need for code changes. Instead we'll use a
configuration file for selecting the data source and plotting library.
{
"data_source": {
"name": "urban_climate_csv"
},
"plot": {
"name": "plotly_plot"
}
}
def test_configure():
app = App.configure(
'config.json'
)
import datetime
import json
class App:
...
@classmethod
26 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
data_source = __import__(config['data_source']
['name']).DataSource()
plot = __import__(config['plot']['name']).Plot()
if __name__ == '__main__':
import sys
from open_weather_json import DataSource
from plotly_plot import Plot
file_name = sys.argv[1]
app = App(DataSource(), Plot())
temperatures_by_hour = app.read(file_name=file_name)
app.draw(temperatures_by_hour)
So, after loading the JSON file, we imported DataSource and Plot from the
respective modules defined in the config file.
import urban_climate_csv
data_source = urban_climate_csv.DataSource()
27 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
collected 6 items
test_app.py ...
[ 42%]
test_matplotlib_plot.py .
[ 57%]
test_open_weather_json.py .
[ 71%]
test_plotly_plot.py .
[ 85%]
test_urban_climate_csv.py .
[100%]
Finally, update the snippet in app.py to use the newly added method:
if __name__ == '__main__':
import sys
config_file = sys.argv[1]
file_name = sys.argv[2]
app = App.configure(config_file)
temperatures_by_hour = app.read(file_name=file_name)
app.draw(temperatures_by_hour)
With the imports eliminated, you can quickly swap one data source or plotting
library for another.
{
"data_source": {
"name": "open_weather_json"
},
"plot": {
"name": "plotly_plot"
}
}
28 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
A Different View
The main App class started as an all-knowing object responsible for reading data
from a CSV and drawing a plot. We used Dependency Injection to decouple the
reading and drawing functionality. The App class is now a container with a simple
interface that connects the reading and drawing parts. The actual reading and
drawing logic is handled in specialized classes that are responsible for one thing
only.
Benefits:
3. Tests doesn't have to change every time that we extend our application
Did we do something special? Not really. The idea behind Dependency Injection
is pretty common in the engineering world, outside of software engineering.
For example, a carpenter who builds house exteriors will generally leave empty
slots for windows and doors so that someone specialized specifically in window
and door installations can install them. When the house is complete and the
owners move in, do they need to tear down half the house just to change an
existing window? No. They can just fix the broken window. As long as the
windows have the same interface (e.g., width, height, depth, etc.), they're able to
install and use them. Can they open the window before it's installed? Of course.
Can they test if the window is broken before they install it? Yes. It's a form of
Dependency Injection too.
Next Steps
1. Extend the app to take a new data source typed called open_weather_api.
This source takes a city, makes the API call, and then returns the data in the
29 of 30
about:reader?url=https://testdriven.io/blog/python-dependency-injection/
Conclusion
2. What are the benefits (and drawbacks) of using Dependency Injection in this
particular area?
If you can easily answer these questions and the benefits outweigh the
drawbacks, go for it. Otherwise, it may not be suitable to use it at the moment.
Happy coding!
30 of 30