Leveraging the power of tempfile library in Python

Create temporary files and directories in Python using tempfile

Photo by Ilya Pavlov on Unsplash

Note: For non-members, this article is also available at https://dineshkumarkb.com/tech/create-temporary-files-and-directories-in-python-using-tempfile/

Occasionally, we may have to create temporary files on our system to write data for a short period of time and then do away with them later. Once such instance is downloading files via http request, do some modifications and upload them to a remote location or like so.

Of course, we could always create a new file or directory and save the file contents into that. However, we may not clean those files which are no longer required.

Python provides a built-in library to create temporary files and directories on all platforms.We could create file like objects that are platform agnostic and also automatically clean up the files once the execution exits the file context manager.

The library provides 4 high-level interfaces for the user to call which can be used as context managers.

1. TemporaryFile
2. NamedTemporaryFile
3. SpooledTemporaryFile
4. TemporaryDirectory

All 4 interfaces do almost the same operation of creating a temporary storage space but with subtle differences.

Let’s quickly look at examples for each.

1.TemporaryFile:

tempfile.TemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None)

This returns a file object for temporary storage purpose. The file is created securely. When I say secure, it means that the file is readable and writable only by the creator i.e the file is accessible only to the user id that created the file.

For consistency of data across platforms, binary mode is used to write to the file. I will save the suffix and prefix params for later.

import tempfile

file = tempfile.TemporaryFile()
with file as f:
    f.write(b'This is a test temp file')
    f.seek(0)
    print(f.read())
Output:
b'This is a test temp file'

If you try to write a file in str mode , you may get the following typeerror.

TypeError: a bytes-like object is required, not 'str'

2. NamedTemporaryFile:

tempfile.NamedTemporaryFile(mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, delete=True, *, errors=None)

The TemporaryFile interface does not have a visible name in the file system. However, the NamedTemporaryFile does have one. We should be able to check the file in the file system.

The filename can be retrieved from the name attribute of the file object.

The suffix and prefix are pretty self-explanatory and are used to provide a suffix and prefix to the file name. This may come in handy if you want to filter temporary files amongst various other files.

If the delete param is set to True, the NamedTemporaryFile acts like a TemporaryFile and immediately deletes the file after its closed. Setting the value to False will persist the temporary file. This can be viewed in the default temporary location saved in the file system.

import tempfile
file = tempfile.NamedTemporaryFile(prefix="tmp_", delete=False)
# prints the prefix of the file
print(tempfile.gettempprefix())
# prints the name of the file with path
print(file.name)
with file as f:
    file.write("This is a new named temporary file".encode())
    file.seek(0)
    print(file.read())

Output:
tmp
/tmp/tmp_8uzye08w
b'This is a new named temporary file'

To get the default temporary directory, use the below method.

import tempfile
print(tempfile.gettempdir())
# When I run the above in ubuntu, I get the below
Output:
/tmp
On windows it will be saved in C:\Users\Temp

3.SpooledTemporaryFile:

tempfile.SpooledTemporaryFile(max_size=0, mode='w+b', buffering=-1, encoding=None, newline=None, suffix=None, prefix=None, dir=None, *, errors=None

This operates similar to TemporaryFile but with one difference. The data is spooled in memory until the file size exceeds max_size. After the file size exceeds max_size the contents are written to the disk.

file = tempfile.SpooledTemporaryFile(max_size=1000)
with file as f:
    file.write("This is a new spooled temporary file".encode())
    file.seek(0)
    print(file.read())

4.TemporaryDirectory:

tempfile.TemporaryDirectory(suffix=None, prefix=None, dir=None)

This function creates a temporary directory exactly like TemporaryFile. After exiting the context, the temporary directory and all of its contents are removed.

import tempfile
folder = tempfile.TemporaryDirectory(prefix="tmp_")
with folder as f:
    print(f" Temp dir created", f)
Output:
Temp dir created /tmp/tmp_ep8qgmc5

The temporary file and temporary directory use mkstemp() and mkdtemp() under the hood respectively.

Summary:

• tempfile can be used to create temporary files and directories in Python
• These are thread-safe and automatically cleaned-up.
• tempfile provides file objects which can be used as context managers.
• NamedTemporaryFile can be persisted by setting delete param to False.
• gettempprefix(), gettempdir() are used to get the prefix and directory of the temporary files respectively.
• The tempfiles are accessible only to the users created the file.

References:

• https://docs.python.org/3/library/tempfile.html