Replace Find Shell commands with Python

April 17, 2023 from Phillip Johnston

I am slowly working on migrating many of my Bash scripts python.

A usual operation that I do is find. Since I have to replace many of them, I would like to document replacement patterns.

Here is an example of A find Command that we will use as a reference:

# First, we need to disable local tracking for build submodules
find src stdlib templates -name "meson" -not -path "*/buildresults/*" -exec sh -c 'git update-index --assume-unchanged {}' \;

I am not An experienced Python developer. These examples may not be optimal. Constructive feedback is welcome.

Pathlib

The best option is Pathlib. The basic setup steps are:

# Including pathlib
from pathlib import Path

# Select a root directory. 
# To mimic find in these examples, we'll use the current directory.
p = Path('.')

Name

Adjusting with names is common find Operation.

The basic approach is the use Path.glob To specify a pattern that can be found. A list of matches is returned. If you want that glob Operation to work recursively, they include ** In your pattern or add recursive=True as a secondary argument for the function. Alternatively you can use Path.rglob without **.

for dir in p.glob('**/meson'):
    print(dir)

You can also use a simpler rating by checking them name Attribute of a file that was returned by a path surgery:

if file.name == 'meson.build'

Search certain directories

Often with findWe would like to limit a search to certain subdirectaries.

There are two main alternatives with which you can achieve this pathlib:

1. Create Path Variables for every subdirectory you want to search and run operations:

   subdir_a = Path('src')
   subdir_b = Path('stdlib')
   subdir_c = Path('templates')
   

2. Carry out glob Operations for every goal subdirectory:

   found_list = list(p.glob('src/**/meson/'))
   found_list += list(p.glob('stdlib/**/meson/'))
   found_list += list(p.glob('templates/**/meson/'))
   

Review

find The operations often state whether they are looking for a file, a symink or a directory. At Patlib you can use Boolean functions to check these details:

• Path.is_dir()
• Path.is_file()
• Path.is_symlink()
• Path.is_symlink()

For example, we can enforce the review meson Be a directory through use path.iterdir():

(dir for dir in ROOT_PATH.iterdir() if dir.is_dir() and dir.name == 'meson')

The problem is that this is not recursive, but only works in the current directory. We need a different approach to scan a tree.

You can use for directories Path.glob with pattern ** (or Path.rglob without **) To check only directories by specifying a subsequent slash:

for dir in p.glob('**/meson/'):
    print(dir)

If you wanted to imitate that iterdir Behavior, you could use a similar structure with glob:

(dir for dir in ROOT_PATH.glob('**/meson') if dir.is_dir())

Without elements

It is also common for us to want find To return certain elements while others are excluded.

In some cases you can use PurePath.match() With a glob-style pattern to carry out the check. You can use this on the full path or you can use it match To PurePath.parent only take into account the list of directors.

# Found list was already grabbed via a glob
found_list = (dir for dir in found_list if not dir.parent.match('*/buildresults'))

One problem here is that we cannot use that ** Style glip agency. If you need more advanced matches according to this line, the best solution I have found is PurePath.as_uri() or PurePath.as_posix() In this way you generate a string and then carry out a standard string.

# Exclude any found results that have `buildresults` as part of the path
found_list = (dir for dir in found_list if not 'buildresults' in dir.as_uri())

Ittery about results

Often with findIf we don’t just generate a list of files, we carry out a certain command. This is as simple as the iterations of the last list and executing the desired operation:

for dir in found_list:
    ctx.run(f"git update-index --assume-unchanged {dir}")

Breaks results

What if you want to collapse the results? This is particularly useful if you want to run a single operation on the full list instead of one.

The easiest way I found is to save the list of results as a string of a string of Posix Path:

found_list = (dir.as_posix() for dir in found_list if not 'buildresults' in dir.as_posix())

Then you can collapse the list with the help .join:

ctx.run(f"git update-index --assume-unchanged {' '.join(found_list)}")

Full replacement example

We referred this sample order at the beginning of the article:

# First, we need to disable local tracking for build submodules
find src stdlib templates -name "meson" -not -path "*/buildresults/*" -exec sh -c 'git update-index --assume-unchanged {}' \;

Here is the complete Python replacement that we came with:

found_list = list(ROOT_PATH.glob('src/**/meson/'))
found_list += list(ROOT_PATH.glob('stdlib/**/meson/'))
found_list += list(ROOT_PATH.glob('templates/**/meson/'))
found_list = (dir.as_posix() for dir in found_list if not 'buildresults' in dir.as_posix())
ctx.run(f"echo git update-index --assume-unchanged {' '.join(found_list)}")

References

Related