0

I want to write unit tests for methods that take a generator as input and produce a generator. I am building on this answer.

For this question let's take the method:

def candidates_for_deletion(self, generations_to_keep=2):
    for files in self.group_by_version(self.backups()).values():
        yield from files[:-generations_to_keep]

I wrote the following test which works, but I don't think I can call it a unit test, as it relies on all methods down to mocked S3 bucket to recreate the data:

import boto3
import pytest
from moto import mock_s3
from gitlabbackups import GitlabBackup


S3_OBJECTS_LIST = [
    '1526781621_2018_05_20_10.8.2_gitlab_backup.tar',
    '1530151221_2018_06_28_11.0.0_gitlab_backup.tar',
    '1530237621_2018_06_29_11.0.0_gitlab_backup.tar',
    '1530324021_2018_06_30_11.0.0_gitlab_backup.tar',
    'some_other_file',
    '1525917621_2018_05_10_10.8.2_gitlab_backup.tar',
    '1530410421_2018_07_01_11.0.1_gitlab_backup.tar',
    '1530496821_2018_07_02_11.0.1_gitlab_backup.tar',
    'a file with spaces inside',
    '1530583221_2018_07_03_11.0.1_gitlab_backup.tar',
    '1530669617_2018_07_04_11.0.1_gitlab_backup.tar'
]


def create_s3_bucket_with_sample_data(bucket):
    conn = boto3.resource('s3', region_name='us-east-1')
    conn.create_bucket(Bucket=bucket)
    for i in S3_OBJECTS_LIST:
        conn.Object(bucket, i).put('some content')


@mock_s3
@pytest.mark.parametrize("generations_to_keep,expected_candidates_for_deletion", [
    (1, ['1525917621_2018_05_10_10.8.2_gitlab_backup.tar',
         '1530151221_2018_06_28_11.0.0_gitlab_backup.tar',
         '1530237621_2018_06_29_11.0.0_gitlab_backup.tar',
         '1530410421_2018_07_01_11.0.1_gitlab_backup.tar',
         '1530496821_2018_07_02_11.0.1_gitlab_backup.tar',
         '1530583221_2018_07_03_11.0.1_gitlab_backup.tar']),
    (2, ['1530151221_2018_06_28_11.0.0_gitlab_backup.tar',
         '1530410421_2018_07_01_11.0.1_gitlab_backup.tar',
         '1530496821_2018_07_02_11.0.1_gitlab_backup.tar'])
])
def test_candidates_for_deletion(generations_to_keep, expected_candidates_for_deletion):
    create_s3_bucket_with_sample_data('mocked-bucket')
    gitlab_backups = GitlabBackups('mocked-bucket')
    candidates_for_deletion = gitlab_backups.candidates_for_deletion(generations_to_keep)
    assert expected_candidates_for_deletion == list(candidates_for_deletion)

I believe I should provide the output of backups(), i.e.

 {'10.8.2': ['1525917621_2018_05_10_10.8.2_gitlab_backup.tar',
             '1526781621_2018_05_20_10.8.2_gitlab_backup.tar'],
  '11.0.0': ['1530151221_2018_06_28_11.0.0_gitlab_backup.tar',
             '1530237621_2018_06_29_11.0.0_gitlab_backup.tar',
             '1530324021_2018_06_30_11.0.0_gitlab_backup.tar'],
  '11.0.1': ['1530410421_2018_07_01_11.0.1_gitlab_backup.tar',
             '1530496821_2018_07_02_11.0.1_gitlab_backup.tar',
             '1530583221_2018_07_03_11.0.1_gitlab_backup.tar',
             '1530669617_2018_07_04_11.0.1_gitlab_backup.tar']}

instead of relying on other methods, but then I am not sure if it should be a helper function returning a generator.

Is there a way to write proper unit tests for such methods or should I rewrite the code?

bumped to the homepage by Community yesterday

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

1

Looks like a usefull test, its possibly an integration test.

That said you should put this in a unitest2.testcase class https://docs.python.org/dev/library/unittest.html

It works nicely with pytest https://docs.pytest.org/en/latest/unittest.html

You can create all your fixtures in the setUpClass method

then all the methods starting with test_ are executed and you can use a nice library of built in assertions.

You you can add your generator generator functions as methods on your TestCase instance then call them from its test_ methods.

Both those links cover the rest of the details.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.