Local Integration Testing With Pytest

2025-09-08T00:00:00+00:00

The traditional testing pyramid encourages us to focus on writing unit tests that are fast and cheap, while avoiding slow and complex integration testing. Real-world experience also shows that poor design and complexity often go hand in hand; I've seen my share of messy and flaky integration test setups. No wonder that people have learned to avoid anything but unit testing.

There is obvious value in verifying how the units of our software integrate to serve use cases. You can't simply ignore integration testing, unless you absolutely love solving production incidents under high pressure. Fortunately, there are ways to test business logic and service integrations in a contained and controlled fashion. If we spend some time and care for the kinds of tests we write, we can achieve robust integration testing without the typical overhead and flakiness associated with higher-level tests.

from celery_decipher.decipher.cipher import ( ROT13_CIPHER, cipher, decipher, ) def test_decipher(): text = "cat" ciphered = cipher(text, ROT13_CIPHER) assert ciphered == "png" deciphered = decipher(ciphered, ROT13_CIPHER) assert deciphered == "cat" </code></pre> The test uses a data fixture and functions imported from one Python module, which is a strong indicator that the test targets tightly related functionality. Well, either that or your modules are organized poorly. We use the public interface of the module, in this case, functions and their signatures, and avoid testing implementation details. If your unit tests break whenever the internal logic in the module changes, you should rethink the modularization of your code, e.g., ensure that the interface is narrow and doesn't leak abstractions.

Component integration tests </a> </h3>

Working within one module is nice and comfortable, but usually doesn't cover many interesting use cases. Unless you inherited the one project, where everything was contained within a single C language function, causing me to lose my faith in humanity. Barring that, you most likely have some sort of a software architecture, where you need to integrate different components to achieve things like sending and receiving messages, uploading files, and querying and persisting data.

a sentiment that many experienced developers agree with</a>. This allows the tests to cover more ground instead of using test doubles for their dependencies.

The example project has a Docker Compose setup for running a local PostgreSQL server with a separate test database. This allows running component integration tests, while a connection to the test database is explicitly injected.

from </span>celery_decipher.decipher.db </span>import </span>(
</span>    get_candidates,
</span>    get_source_text,
</span>    insert_source_text,
</span>)
</span>from </span>celery_decipher.decipher.solver </span>import </span>(
</span>    </span>POPULATION_SIZE</span>,
</span>    initial_guess,
</span>)
</span>
</span>def </span>test_initial_guess</span>(</span>testdb_cursor</span>):
</span>    text = "</span>Smoky smoke test</span>"
</span>    source_text_id = </span>insert_source_text</span>(testdb_cursor, text)
</span>    </span>initial_guess</span>(testdb_cursor, source_text_id)
</span>    candidates = </span>get_candidates</span>(testdb_cursor, source_text_id)
</span>    </span>assert </span>candidates is not </span>None
</span>    </span>assert </span>len</span>(candidates) == </span>POPULATION_SIZE
</span></code></pre>
This example test uses database layer functions for inserting test data and fetching the results after calling the higher-level function which we are testing. When the modules are deep and their interfaces are narrow, our tests are clean: We don't need to care how the higher-level function initial_guess</code> does its magic, while it internally uses the same database module as we do. In fact, we have no business in looking inside initial_guess</code>, for example, poking for state changes within the solver</code> module. We should test the public interface instead, while avoiding assumptions about implementation details. In general, you shouldn't create tight couplings in tests, so that you can benefit from loose couplings between modules.</p>
Granted, we do have to know that initial_guess</code> reads the source text from the database and stores results, which we query using the get_candidates</code> function. However, this doesn't violate the aforementioned principle per se, but is a part of the public interface, that is, the contract between the function and its caller. We should aim to keep this interface as narrow as possible and the contracts between modules clear. In the majority of cases, the interface should cover just the function signature, i.e., arguments to the function and what it returns. Nonetheless, in other cases allowing our contract to cover effects in databases and other side-effects doesn't add too much to the complexity. In our example, the affected database is provided as a parameter to the function call, so we're still being explicit about it.</p>
In our walk up the pyramid, we are moving to higher and higher levels of abstraction. Whereas unit tests could validate internal logic within a module, component integration tests avoid looking into individual modules, test higher-level interfaces, and validate that components work together as intended. When we move to the next level in our hierarchy, we no longer care about the individual components at all, but treat the overall system as a black box.</p>
API layer tests</a>
</h3>
The next layer of tests shifts our focus from the internal logic of a service to its external interface. In web services, this usually means testing the HTTP API layer. Martin Fowler has called such tests "subcutaneous"</a>, as the API layer sits just beneath the UI surface in a web application.</p>
The API layer tests validate that the service adheres to its contract, i.e., it handles requests appropriately and gives valid responses. In my example project, requests and responses are validated using Pydantic models</a>.</p>
from </span>uuid </span>import </span>UUID
</span>from </span>celery_decipher.decipher.models </span>import </span>DecipherStatusResponse
</span>
</span>def </span>test_ingest</span>(</span>http_client</span>):
</span>    text = "</span>Smoky smoke test</span>"
</span>    response = http_client.</span>post</span>("</span>/decipher</span>", </span>json</span>={"</span>text</span>": text})
</span>    </span>assert </span>response.status_code == </span>200
</span>    source_text_id = </span>UUID</span>(response.</span>json</span>()["</span>source_text_id</span>"])
</span>
</span>    status_response = http_client.</span>get</span>(</span>f</span>"</span>/decipher/</span>{source_text_id}")
</span>    </span>assert </span>status_response.status_code == </span>200
</span>    status = DecipherStatusResponse.</span>model_validate</span>(status_response.</span>json</span>())
</span>    </span>assert </span>status.source_text_id == source_text_id
</span>    </span>assert </span>status.source_text == text
</span>    </span>assert </span>status.status == "</span>PENDING</span>"
</span></code></pre>
In the example above, the flow of the test is simple. Our service endpoint accepts a text to be deciphered, we receive an unique ID for the request, and we can query the status of the request using that ID. We don't know how the service manages these tasks internally and what dependencies, if any, it has for performing the actual deciphering. The API covers the internal logic, providing a shared contract between the service and its clients, while allowing us to treat the service as a black box.</p>
Often the features tested in the API layer require the service to call other services, external dependencies, databases, and so on. Sometimes, you may be able to set up a test environment for all the dependencies, and verify the full flow of the feature. Other times, it's not practical for reasons like legacy services without test environments, third-party services with restrictive licensing, complex and time-consuming operations, and so on. In such cases, you can use test doubles for the dependencies, utilizing fakes, stubs and mocks as appropriate. Just be careful! The more you mock, the more complex your test setup becomes, until you end up verifying the mocks instead of the service</strong>.</p>
End-to-end tests</a>
</h3>
Finally, we reach the top of the pyramid, where we test complete features and use cases. In web applications, this usually means driving the UI using tools like Selenium, Playwright, or Cypress. In other applications, it may involve prompting the system from upstream services, triggering processing by sending events to message queues, and so on. The difference to API layer tests is the coverage of entire use cases, which often involve multiple services and external systems.</p>
The upstream services may provide a natural way of receiving the results for some use cases, for example, by sending an asynchronous task to an event queue and reading the final result from another queue, or by calling a REST API endpoint and receiving a HTTP response with synchronous results. Other use cases may produce results in an external system, for example, by creating files in a shared storage, sending emails, or updating records in a third-party system. Getting and verifying the results from an external system, and then cleaning them up afterwards, may sometimes be tricky. The required effort is partly why E2E tests can be slow and expensive to maintain.</p>
The example project does not have a frontend or an UI of any kind. The use cases are fully covered by the API layer tests, as the HTTP endpoints implement complete features. The web application drives all the use cases, functioning as the upstream and interface layer. If your application is structured this way, then testing it is straight-forward and simple. However, most real-world systems I've worked with have had multiple entrypoints for different use cases, for example, a web UI for interactive use, REST API endpoints for programmatic access, and batch data processing jobs triggered when data is received from integrations. Covering all these diverse use cases requires significant effort, since we can't apply a single template to all of them.</p>
Final thoughts</a>
</h2>
We need tests. They allow me to sleep at night, knowing that my code works as intended. They document how functions behave, how use cases are implemented, and what are the contracts between the components of my system. They allow me to refactor and continuously improve my code, while maintaining quality and mitigating the risk of breaking things.</p>
We need layered tests. Unit tests are great for verifying internal logic of its components, while component integration tests ensure that the components work together. Subcutaneous API layer tests validate the interface contracts, while ensuring appropriate level of abstraction that hides implementation details. Finally, E2E tests cover entire use cases, which ultimately ensure that our software provides value to its users.</p>
We need to learn to love testing. It is a skill that requires practice, identifying patterns and matching them to appropriate test strategies. It should be a natural part of development, whether you're doing test-driven development or not.</p>

Google Cloud Storage as S3 - Case Joplin

2025-03-23T00:00:00+00:00

I use Joplin</a> as my note-taking app over multiple devices, which makes a shared backend for note-syncing a thing I can't live without. Joplin supports a ton of different backends, including AWS S3. Obviously, as a Google enthusiast, I really really wanted to use Google Cloud Storage as my S3 backend. This was a surprisingly easy thing to do.

Recently Google Cloud has been my first choice for personal cloud projects. It feels like a very developer-centric platform with developer-centric tooling. Arguably, Google seems to think that everyone working in technology is a developer</a>. This is a sentiment I can get behind, as I aspire to be more of a hacker than an enterprise user.

For this curious case of using Cloud Storage in place of S3, Google has built an XML interoperability layer</a>. Getting Joplin to talk to this API was as simple as providing the Cloud Storage URI https://storage.googleapis.com</code> in place of the S3 URL.

TL;DR: You can find the complete Terraform configuration for this setup in this Gist</a>. Following is a breakdown of the key resources and why you need them.

Bucket</a>
</h2>
First, we need a Cloud Storage bucket for the synchronized Joplin state. I like to use a random suffix to ensure the bucket name is unique, as in the following Terraform snippet.</p>
resource </span>"</span>random_id</span>" "</span>bucket-suffix</span>" {
</span>  </span>byte_length </span>= </span>8
</span>  </span>keepers </span>= {
</span>    </span>project </span>= </span>var</span>.</span>project
</span>  }
</span>}
</span>
</span>resource </span>"</span>google_storage_bucket</span>" "</span>sync</span>" {
</span>  </span>name                     </span>= "</span>joplin-</span>${</span>random_id</span>.</span>bucket-suffix</span>.</span>hex</span>}"
</span>  </span>public_access_prevention </span>= "</span>enforced</span>"
</span>  </span>location                 </span>= </span>var</span>.</span>region
</span>  </span>project                  </span>= </span>var</span>.</span>project
</span>}
</span></code></pre>
Note that I am using Terraform variables project</code> and region</code> throughout these examples.</p>
Service Account</a>
</h2>
Then, I recommend creating a separate service account with minimum permissions to access the bucket. Good infosec hygiene keeps the setup less smelly.</p>
resource </span>"</span>google_service_account</span>" "</span>sync</span>" {
</span>  </span>account_id   </span>= "</span>sync-bucket</span>"
</span>  </span>display_name </span>= "</span>Joplin sync bucket service account</span>"
</span>}
</span>
</span>resource </span>"</span>google_storage_bucket_iam_member</span>" "</span>sync</span>" {
</span>  </span>bucket </span>= </span>google_storage_bucket</span>.</span>sync</span>.</span>name
</span>  </span>role   </span>= "</span>roles/storage.objectUser</span>"
</span>  </span>member </span>= "</span>serviceAccount:</span>${</span>google_service_account</span>.</span>sync</span>.</span>email</span>}"
</span>}
</span></code></pre>
roles/storage.objectUser</code> gives the account permission to do what it likes with objects and folders</a> in the bucket, without allowing it to modify access and permissions. Joplin likes to create and delete both objects and folders, so this is the best fit from predefined roles for GCS.</p>
Access key and secret</a>
</h2>
Access to S3 is authenticated with an access key and secret. We can generate HMAC keys for the service account and push them to Secrets Manager.</p>
resource </span>"</span>google_storage_hmac_key</span>" "</span>sync</span>" {
</span>  </span>service_account_email </span>= </span>google_service_account</span>.</span>sync</span>.</span>email
</span>}
</span>
</span>resource </span>"</span>google_secret_manager_secret</span>" "</span>hmac-secret</span>" {
</span>  </span>secret_id </span>= "</span>hmac-secret</span>"
</span>  replication {
</span>    auto {}
</span>  }
</span>}
</span>
</span>resource </span>"</span>google_secret_manager_secret</span>" "</span>hmac-id</span>" {
</span>  </span>secret_id </span>= "</span>hmac-id</span>"
</span>  replication {
</span>    auto {}
</span>  }
</span>}
</span>
</span>resource </span>"</span>google_secret_manager_secret_version</span>" "</span>hmac-secret</span>" {
</span>  </span>secret      </span>= </span>google_secret_manager_secret</span>.</span>hmac-secret</span>.</span>id
</span>  </span>secret_data </span>= </span>google_storage_hmac_key</span>.</span>sync</span>.</span>secret
</span>}
</span>
</span>resource </span>"</span>google_secret_manager_secret_version</span>" "</span>hmac-id</span>" {
</span>  </span>secret      </span>= </span>google_secret_manager_secret</span>.</span>hmac-id</span>.</span>id
</span>  </span>secret_data </span>= </span>google_storage_hmac_key</span>.</span>sync</span>.</span>access_id
</span>}
</span></code></pre>
Joplin configuration</a>
</h2>
Finally, we can configure Joplin</a> to use the bucket and HMAC key. The Google parameters match S3 settings pretty closely.</p>

S3 bucket</strong>: output of google_storage_bucket.sync.name</code>.</li>
S3 URL</strong>: storage.googleapis.com</code> (Google's XML API).</li>
S3 region</strong>: your var.region</code> (Google Cloud region).</li>
The S3 access key</strong>: HMAC ID (fetch from Secrets Manager, e.g., gcloud secrets versions access latest --secret hmac-id</code>).</li>
The S3 access secret</strong>: HMAC secret (fetch from Secrets Manager).</li>
</ul>

Blog posts - Toni Vanhala

Local Integration Testing With Pytest

End-to-end testsFinal thoughts

Final thoughts

Google Cloud Storage as S3 - Case Joplin