Getting warning from unit test

Question

I get a warning from this test but I don't know how to handle it. The warning is

/usr/lib/python3.5/unittest/case.py:600: ResourceWarning: unclosed file <_io.BufferedReader name='/tmp/20170926_084744_545761.pdf'> testMethod()

The code I'm testing (with Mako and Bottle) is:

  pdf.pdf_export(selected_date=selected_date, values=values, to_pdf_file=to_pdf_file)
            return static_file(to_pdf_file, root='/tmp', download=filename)

My test case looks as follows

 # Test is based on assumption of having the appropriate parameters for the expected output
    def testPDFexportByLogic(self):
        filter = ''
        sortby = '1'
        reverse1 = '1'
        reverse = True
        if reverse1 == '-1': reverse = False

        oncall_logic = OncallLogic(self.resources, view_type=OncallViewType.daily)
        oncall_logic.set_selections(selected_customer_id='417', listing_customer_ids='417',
                                    selected_date='20170925')
        pdf_file = oncall_logic.get_oncall_pdf_export_daily_events(self.resources, '417',
                                                                   '417', '20170925', filter=filter,
                                                                   sortby=int(sortby), reverse=reverse)

        filename = 'pdf-temp'
        with open(filename, 'wb') as w:
            w.write(pdf_file.body.read())
        self.assertIn("Sjukhus", str(self.convert_pdf_to_txt(filename)))
        os.remove(filename)

The above test succeeds to render the PDF and check the text, but I get a warning from running the test. Is the warning from the actual test or from running the code which is tested? Which file is it that I must close, if any?

I restructure the code and I still get the warning. I still get the ResourceWarning. I don't understand it because I only open one file and the warning is about the other file. Why???

def testPDFexportByLogic(self):

    filename = 'pdf-temp'
    with open(filename, 'wb') as w:
        oncall_logic = OncallLogic(self.resources, view_type=OncallViewType.daily)
        oncall_logic.set_selections(selected_customer_id='417', listing_customer_ids='417',
                                    selected_date='20170925')
        pdf_file = oncall_logic.get_oncall_pdf_export_daily_events(self.resources, '417',
                                                                   '417', '20170925')
        w.write(pdf_file.body.read())
        w.close()
        self.assertIn("Sjukhus", str(self.convert_pdf_to_txt(filename)))

These are my import statements in the test code:

import os
import io
import pdfminer

from controllers.oncall import oncall
from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
from pdfminer.converter import TextConverter
from pdfminer.layout import LAParams
from pdfminer.pdfpage import PDFPage
from business.logic.oncall.oncall_logic import OncallLogic
from business.resources.entities.gui.gui_settings import OncallViewType

The following is more code that is tested:

def pdf_export(self, selected_date, values, to_pdf_file):
    """
     @param selected_date: Date of the view to export
     @type selected_date: str
     @param values: Dict with values for the template the write on
     @type values: dict
     @param to_pdf_file: Name of the PDF file to write
     @type to_pdf_file: str
     """
    css = '/srv/http/python.dev/python-common/static_files/css/pdf-export-styles.css'
    pdfkit.from_string(render_template("oncall/dayview_pdf_export", values), '/tmp/' + to_pdf_file, css=css)

Answer 1

Summing up what we had debugged and figured out:

• the problem comes from the static_file() function in the bottle library

• if we look at the source code of that static_file() function, we can find this line:

  body = '' if request.method == 'HEAD' else open(filename, 'rb')
  

  which uses the "unsafe" open() call without explicitly closing the file.

Bottle had a similar problem in the other part of the framework before. Just reported this problem as a new issue:

• ResourceWarning: unclosed file <_io.BufferedReader... in static_file()

And, until it is fixed, you may ignore the warning since this is not coming from your code and is not an error.

Answer 2

As per the explanation given here

This ResourceWarning means that you opened a file, used it, but then forgot to close the file. Python closes it for you when it notices that the file object is dead, but this only occurs after some unknown time has elapsed. Thus in recent versions, Python also prints a ResourceWarning when it does that. It is a way for you to quickly identify where the unclosed files are, and properly close them. It might be important on some platforms which cannot have more than N files opened at the same time (e.g. 1024). Also, specifically on Windows, you cannot do some operations with a file if it's still open (e.g. deleting it).

You should structure the code so that the work and conversion/assertion you're doing is inside the with open(filename, 'wb') as w: block. You can take a cue from how the code is structured here.