How do I recursively grep all directories and subdirectories?

Asked 2023-09-20 20:56:30 View 656,853

How do I recursively grep all directories and subdirectories?

find . | xargs grep "texthere" *
  • @TC1 The sad thing is that grep itself can answer the question (at least GNU grep): grep --help |grep recursive - anyone
  • If you find yourself frequently using grep to do recursive searches (especially if you manually do a lot of file/directory exlusions), you may find ack (a very programmer-friendly grep alternative) useful. - anyone
  • Actually neither -r nor --recursive work on the Solaris box I use at work. And the man page for grep doesn't mention anything recursive. I had to resort to find and xargs myself. - anyone
  • ag is my favorite way to do this now github.com/ggreer/the_silver_searcher - anyone
  • grep -rin xlsx *.pl doesn't work for me on Redhat Linux. I get a "no match" error. - anyone

Answers

grep -r "texthere" .

The first parameter represents the regular expression to search for, while the second one represents the directory that should be searched. In this case, . means the current directory.

Note: This works for GNU grep, and on some platforms like Solaris you must specifically use GNU grep as opposed to legacy implementation. For Solaris this is the ggrep command.

Answered   2023-09-20 20:56:30

  • Note: "grep -r" only works on newer greps. It doesn't work on the grep that comes with AIX 5.3 for example. - anyone
  • Use grep -R to follow symlinks. - anyone
  • It is good to know that "-i" would make it case insensitive, and "-n" also include the line number for each matched result. - anyone
  • also good to know, if you are just looking for a fixed string and not a regex, use -F option. it will save you scads of time by not invoking the regex parser. very handy if you are searching lots of files. - anyone
  • alias rgrep='grep -r' - anyone

If you know the extension or pattern of the file you would like, another method is to use --include option:

grep -r --include "*.txt" texthere .

You can also mention files to exclude with --exclude.

Ag

If you frequently search through code, Ag (The Silver Searcher) is a much faster alternative to grep, that's customized for searching code. For instance, it's recursive by default and automatically ignores files and directories listed in .gitignore, so you don't have to keep passing the same cumbersome exclude options to grep or find.

Answered   2023-09-20 20:56:30

  • Works great with grep that comes with Linux & Cygwin, but not with the one that comes with AIX. - anyone
  • @KrzysztofWolny: ` ` instead of = works just fine on Ubuntu. PS: that's supposed to be a backticked space, but the SO markdown parser failed. - anyone
  • @DanDascalescu I upvoted for the grep, not for the Ag, just so you know :) - anyone
  • Do we have an option to exclude a directory while searching recursively? - anyone
  • Windows cygwin likes double-quotes --include "*.txt" --include "*.TXT" - anyone

I now always use (even on Windows with GoW -- Gnu on Windows):

grep --include="*.xxx" -nRHI "my Text to grep" *

(As noted by kronen in the comments, you can add 2>/dev/null to void permission denied outputs)

That includes the following options:

--include=PATTERN

Recurse in directories only searching file matching PATTERN.

-n, --line-number

Prefix each line of output with the line number within its input file.

(Note: phuclv adds in the comments that -n decreases performance a lot so, so you might want to skip that option)

-R, -r, --recursive

Read all files under each directory, recursively; this is equivalent to the -d recurse option.

-H, --with-filename

Print the filename for each match.

-I     

Process a binary file as if it did not contain matching data;
this is equivalent to the --binary-files=without-match option.

And I can add 'i' (-nRHIi), if I want case-insensitive results.

I can get:

/home/vonc/gitpoc/passenger/gitlist/github #grep --include="*.php" -nRHI "hidden" *
src/GitList/Application.php:43:            'git.hidden'      => $config->get('git', 'hidden') ? $config->get('git', 'hidden') : array(),
src/GitList/Provider/GitServiceProvider.php:21:            $options['hidden'] = $app['git.hidden'];
tests/InterfaceTest.php:32:        $options['hidden'] = array(self::$tmpdir . '/hiddenrepo');
vendor/klaussilveira/gitter/lib/Gitter/Client.php:20:    protected $hidden;
vendor/klaussilveira/gitter/lib/Gitter/Client.php:170:     * Get hidden repository list
vendor/klaussilveira/gitter/lib/Gitter/Client.php:176:        return $this->hidden;
...

Answered   2023-09-20 20:56:30

  • Gow looks promising - newer than the GNU Windows utilities that I have been using. Trying it now... - anyone
  • what is the meaning of the last character * here? - anyone
  • @lorniper it makes the shell select all files and folders in your current directory, making in turn the grep apply to those files and (recursively because of the -R option) to the folders. - anyone
  • @lorniper Noy exactly: * or . is a glob pattern (interpreted by the shell): unix.stackexchange.com/a/64695/7490. '.' will select dotfiles or dot folders as well (like .git/) - anyone
  • previously I've always used grep -rnI but then I learned that -n decreases performance a lot so I just use it when really needed and normally I'll use -rI - anyone

Also:

find ./ -type f -print0 | xargs -0 grep "foo"

but grep -r is a better answer.

Answered   2023-09-20 20:56:30

  • Or if you don't want to worry about spaces in filenames find . -type f -exec grep "foo" '{}' \; works well where supported. - anyone
  • If you are going to pipe find through xargs to grep, AND if you are only searching for a fixed string (i.e., not a regex), you might benefit from invoking the grep -F option, so grep won't load the regex engine for each invocation. If there are a lot of files it will be much faster. - anyone
  • find . -type f -exec grep -Hu "foo" {} \; is what I use as it gives the filename. - anyone
  • This works on all *nix because it is POSIX 7 - anyone
  • find ./ -type f -print0 | xargs -0 grep "foo" - anyone

globbing **

Using grep -r works, but it may overkill, especially in large folders.

For more practical usage, here is the syntax which uses globbing syntax (**):

grep "texthere" **/*.txt

which greps only specific files with pattern selected pattern. It works for supported shells such as Bash +4 or zsh.

To activate this feature, run: shopt -s globstar.

See also: How do I find all files containing specific text on Linux?

git grep

For projects under Git version control, use:

git grep "pattern"

which is much quicker.

ripgrep

For larger projects, the quickest grepping tool is ripgrep which greps files recursively by default:

rg "pattern" .

It's built on top of Rust's regex engine which uses finite automata, SIMD and aggressive literal optimizations to make searching very fast. Check the detailed analysis here.

Answered   2023-09-20 20:56:30

  • Thanks for the git grep suggestion -- it is very useful and I did not know about it! - anyone
  • Thanks for the ripgrep suggestion. It's way faster. - anyone

In POSIX systems, you don't find -r parameter for grep and your grep -rn "stuff" . won't run, but if you use find command it will:

find . -type f -exec grep -n "stuff" {} \; -print

Agreed by Solaris and HP-UX.

Answered   2023-09-20 20:56:30

  • what is the meaning of {} \; -print respectively? - anyone
  • In -exec option - symbol {} is a reference to the filename which currently found by find tool (that is to do something with the filename we found), also -exec option should be terminated with ; symbol (to mark ending of the exec commands), but because this is all running in a shell that symbol should be escaped.. and finally -print option lets find tool to print out found filenames on the screen. - anyone

If you only want to follow actual directories, and not symbolic links,

grep -r "thingToBeFound" directory

If you want to follow symbolic links as well as actual directories (be careful of infinite recursion),

grep -R "thing to be found" directory

Since you're trying to grep recursively, the following options may also be useful to you:

-H: outputs the filename with the line

-n: outputs the line number in the file

So if you want to find all files containing Darth Vader in the current directory or any subdirectories and capture the filename and line number, but do not want the recursion to follow symbolic links, the command would be

grep -rnH "Darth Vader" .

If you want to find all mentions of the word cat in the directory

/home/adam/Desktop/TomAndJerry 

and you're currently in the directory

/home/adam/Desktop/WorldDominationPlot

and you want to capture the filename but not the line number of any instance of the string "cats", and you want the recursion to follow symbolic links if it finds them, you could run either of the following

grep -RH "cats" ../TomAndJerry                   #relative directory

grep -RH "cats" /home/adam/Desktop/TomAndJerry   #absolute directory

Source:

running "grep --help"

A short introduction to symbolic links, for anyone reading this answer and confused by my reference to them: https://www.nixtutor.com/freebsd/understanding-symbolic-links/

Answered   2023-09-20 20:56:30

  • Great answer. The extra switches (-rnh) are very helpful, so thanks for suggesting them. - anyone

To find name of files with path recursively containing the particular string use below command for UNIX:

find . | xargs grep "searched-string"

for Linux:

grep -r "searched-string" .

find a file on UNIX server

find . -type f -name file_name

find a file on LINUX server

find . -name file_name

Answered   2023-09-20 20:56:30

just the filenames can be useful too

grep -r -l "foo" .

Answered   2023-09-20 20:56:30

another syntax to grep a string in all files on a Linux system recursively

grep -irn "string"

the -r indicates a recursive search that searches for the specified string in the given directory and sub directory looking for the specified string in files, program, etc

-i ingnore case sensitive can be used to add inverted case string

-n prints the line number of the specified string

NB: this prints massive result to the console so you might need to filter the output by piping and remove less interesting bits of info. It also searches binary programs so you might want to filter some of the results

Answered   2023-09-20 20:56:30

ag is my favorite way to do this now github.com/ggreer/the_silver_searcher . It's basically the same thing as ack but with a few more optimizations.

Here's a short benchmark. I clear the cache before each test (cf https://askubuntu.com/questions/155768/how-do-i-clean-or-disable-the-memory-cache )

ryan@3G08$ sync && echo 3 | sudo tee /proc/sys/vm/drop_caches
3
ryan@3G08$ time grep -r "hey ya" .

real    0m9.458s
user    0m0.368s
sys 0m3.788s
ryan@3G08:$ sync && echo 3 | sudo tee /proc/sys/vm/drop_caches
3
ryan@3G08$ time ack-grep "hey ya" .

real    0m6.296s
user    0m0.716s
sys 0m1.056s
ryan@3G08$ sync && echo 3 | sudo tee /proc/sys/vm/drop_caches
3
ryan@3G08$ time ag "hey ya" .

real    0m5.641s
user    0m0.356s
sys 0m3.444s
ryan@3G08$ time ag "hey ya" . #test without first clearing cache

real    0m0.154s
user    0m0.224s
sys 0m0.172s

Answered   2023-09-20 20:56:30

This should work:

grep -R "texthere" *

Answered   2023-09-20 20:56:30

If you are looking for a specific content in all files from a directory structure, you may use find since it is more clear what you are doing:

find -type f -exec grep -l "texthere" {} +

Note that -l (downcase of L) shows the name of the file that contains the text. Remove it if you instead want to print the match itself. Or use -H to get the file together with the match. All together, other alternatives are:

find -type f -exec grep -Hn "texthere" {} +

Where -n prints the line number.

Answered   2023-09-20 20:56:30

  • Up-voted for being the only find solution to both avoid unnecessary use of xargs and use + instead of \; with -exec, thereby avoiding tons of unnecessary process launches. :-) - anyone

This is the one that worked for my case on my current machine (git bash on windows 7):

find ./ -type f -iname "*.cs" -print0 | xargs -0 grep "content pattern"

I always forget the -print0 and -0 for paths with spaces.

EDIT: My preferred tool is now instead ripgrep: https://github.com/BurntSushi/ripgrep/releases . It's really fast and has better defaults (like recursive by default). Same example as my original answer but using ripgrep: rg -g "*.cs" "content pattern"

Answered   2023-09-20 20:56:30

grep -r "texthere" . (notice period at the end)

(^credit: https://stackoverflow.com/a/1987928/1438029)


Clarification:

grep -r "texthere" / (recursively grep all directories and subdirectories)

grep -r "texthere" . (recursively grep these directories and subdirectories)

grep recursive

grep [options] PATTERN [FILE...]

[options]

-R, -r, --recursive

Read all files under each directory, recursively.

This is equivalent to the -d recurse or --directories=recurse option.

http://linuxcommand.org/man_pages/grep1.html

grep help

$ grep --help

$ grep --help |grep recursive
  -r, --recursive           like --directories=recurse
  -R, --dereference-recursive

Alternatives

ack (http://beyondgrep.com/)

ag (http://github.com/ggreer/the_silver_searcher)

Answered   2023-09-20 20:56:30

Throwing my two cents here. As others already mentioned grep -r doesn't work on every platform. This may sound silly but I always use git.

git grep "texthere"

Even if the directory is not staged, I just stage it and use git grep.

Answered   2023-09-20 20:56:30

Below are the command for search a String recursively on Unix and Linux environment.

for UNIX command is:

find . -name "string to be searched" -exec grep "text" "{}" \;

for Linux command is:

grep -r "string to be searched" .

Answered   2023-09-20 20:56:30

  • The use of find with -exec needs more upvotes over the numerous answers with the non-portable and (slightly) less efficient -print0 | xargs -0 - anyone

In 2018, you want to use ripgrep or the-silver-searcher because they are way faster than the alternatives.

Here is a directory with 336 first-level subdirectories:

% find . -maxdepth 1 -type d | wc -l
     336

% time rg -w aggs -g '*.py'
...
rg -w aggs -g '*.py'  1.24s user 2.23s system 283% cpu 1.222 total

% time ag -w aggs -G '.*py$'
...
ag -w aggs -G '.*py$'  2.71s user 1.55s system 116% cpu 3.651 total

% time find ./ -type f -name '*.py' | xargs grep -w aggs
...
find ./ -type f -name '*.py'  1.34s user 5.68s system 32% cpu 21.329 total
xargs grep -w aggs  6.65s user 0.49s system 32% cpu 22.164 total

On OSX, this installs ripgrep: brew install ripgrep. This installs silver-searcher: brew install the_silver_searcher.

Answered   2023-09-20 20:56:30

  • Speed is important if you need to do this often, but most of us find ourselves only doing this a few times a year at most. Installing the latest spiffy third-party juju tool du jour is overkill and the solutions which haven't changed much since 1978 are good to know regardless. - anyone
  • I find it highly implausible that a programmer would search for text in a source tree only several times per year. But even from the viewpoint of usability, rg has a considerable edge over cobbling together a recursive grep command from scratch. Using rg: rg foo. Using unix tools: find . | xargs grep foo. And if any of your files has a quote in it, you need to use find . -print0 | xargs -0 grep foo. Are you going to remember that if you use this a few times a year? - anyone
  • You're forgetting find . -type f -exec grep 'regex' {} + which indeed is easy to remember if you use these tools with any regularity. But probably you should run ctags or etags on your source tree anyway if you need to find stuff frequently. - anyone
  • I've been using ripgrep and it's great. But silver searcher is fantastic for programmers. +1 - anyone

In my IBM AIX Server (OS version: AIX 5.2), use:

find ./ -type f -print -exec grep -n -i "stringYouWannaFind" {} \; 

this will print out path/file name and relative line number in the file like:

./inc/xxxx_x.h

2865: /** Description : stringYouWannaFind */

anyway,it works for me : )

Answered   2023-09-20 20:56:30

For a list of available flags:

grep --help 

Returns all matches for the regexp texthere in the current directory, with the corresponding line number:

grep -rn "texthere" .

Returns all matches for texthere, starting at the root directory, with the corresponding line number and ignoring case:

grep -rni "texthere" /

flags used here:

  • -r recursive
  • -n print line number with output
  • -i ignore case

Answered   2023-09-20 20:56:30

Note that find . -type f | xargs grep whatever sorts of solutions will run into "Argument list to long" errors when there are too many files matched by find.

The best bet is grep -r but if that isn't available, use find . -type f -exec grep -H whatever {} \; instead.

Answered   2023-09-20 20:56:30

  • Huh? xargs is specifically a workaround for the "Argument list too long" problem. - anyone
  • Well, no - xargs is specifically for converting a pipe of arguments to an arglist, but yes, it is true that modern xargs when used with -s and/or -L can deal with very long arglists by breaking into multiple command invocations, but it isn't configured that way by default (and wasn't in any of the above responses). As an example: find . -type f | xargs -L 100 grep whatever - anyone
  • Which platform would that be on? POSIX xargs is standardized to have this behavior out of the box. "The xargs utility shall limit the command line length such that when the command line is invoked, the combined argument and environment lists ... shall not exceed {ARG_MAX}-2048 bytes." - anyone
  • Hm. While the gnu docs are less clear than posix on this basis, and I no longer have access to the machine that caused me to make this statement, I cannot confirm my original interpretation on any current implementation. Recursive grep is, of course, still preferable if available, but there's little reason to avoid the xargs recipe (do use -H for the grep to avoid the final invocation of grep getting passed only a single filename, though). - anyone

I guess this is what you're trying to write

grep myText $(find .)

and this may be something else helpful if you want to find the files grep hit

grep myText $(find .) | cut -d : -f 1 | sort | uniq

Answered   2023-09-20 20:56:30

Here's a recursive (tested lightly with bash and sh) function that traverses all subfolders of a given folder ($1) and using grep searches for given string ($3) in given files ($2):

$ cat script.sh
#!/bin/sh

cd "$1"

loop () {
    for i in *
    do
        if [ -d "$i" ]
        then
            # echo entering "$i"
            cd "$i"
            loop "$1" "$2"
        fi
    done

    if [ -f "$1" ]
    then
        grep -l "$2" "$PWD/$1"
    fi

    cd ..
}

loop "$2" "$3"

Running it and an example output:

$ sh script start_folder filename search_string
/home/james/start_folder/dir2/filename

Answered   2023-09-20 20:56:30

For .gz files, recursively scan all files and directories Change file type or put *

find . -name \*.gz -print0 | xargs -0 zgrep "STRING"

Answered   2023-09-20 20:56:30

Just for fun, a quick and dirty search of *.txt files if the @christangrant answer is too much to type :-)

grep -r texthere .|grep .txt

Answered   2023-09-20 20:56:30

Get the first matched files from grep command and get all the files don't contain some word, but input files for second grep comes from result files of first grep command.

grep -l -r --include "*.js" "FIRSTWORD" * | xargs grep "SECONDwORD"
grep -l -r --include "*.js" "FIRSTWORD" * | xargs grep -L "SECONDwORD"

dc0fd654-37df-4420-8ba5-6046a9dbe406

grep -l -r --include "*.js" "SEARCHWORD" * | awk -F'/' '{print $NF}' | xargs -I{} sh -c 'echo {}; grep -l -r --include "*.html" -w --include=*.js -e {} *;  echo '''

5319778a-cec2-444d-bcc4-53d33821fedb

grep "SEARCH_STRING" *.log | grep -e "http" -e "https" | awk '{print $NF}' | uniq

ce91d131-a5c2-4cc8-b836-1461feee6cdb

Here's how you can modify the command to extract the value of messageName:

grep -m 2 "In sendMessage:: " *LOGFILE.log | grep -o -e "messageName=[^,]*" | cut -d= -f2 | sort | uniq | tee >(echo "Number of unique values: $(wc -l)")

grep "In Message:: " *messaging.log | grep -o -e "messageName=[^,]*" | cut -d= -f2 | sort | uniq | while read -r messageName; do grep -m 1 "In  sendMessage:: .*messageName=${messageName}" *logfile.log | head -n 1; done

I want to use run below grep command over above files 2. files to be sorted in descending order based on their update time and not matching the .gz format

grep "org.springframework.batch.item.ItemStreamException: Failed to initialize the reader at" $(ls -lrth | grep -i opti | awk '{print $NF}')
      grep -A 15 "request to URL : SEARCH" $(ls -lth | grep "common" | grep -v ".gz"  | awk '{print $NF}')

command to create a new file from the first occurrence to the last occurrence.

sed -n '/14 Jan 2023/,/14 Jan 2023/p' common.log > common_1day.log

Today modified files,

ls -lrth $(find . -type f -name "*.log" -newermt "$(date -R -d 'today 00:00')" -print)
grep "CID" $(find . -type f -name "*.log" -newermt "$(date -R -d 'today 00:00')" -print)
zgrep "SEARCH" $(find . -type f -newermt "$(date -R -d 'today 00:00')" -print)
ls -lrth $(find . -type f -name "*" -newermt "$(date -R -d 'today 00:00')" -print)
less +G $(find . -type f -name "*LOG_FILE.log" -newermt "$(date -R -d 'today 00:00')" -print)
grep Async $(find . -type f -name "*" -newermt "2023-04-14 00:00:00" ! -newermt "2023-04-16 00:00:00" -print)

Find commands

find . -type f -not -path "*/target/*" -name "log4j2.xml" -exec grep -H '<Async name="' {} \;

Answered   2023-09-20 20:56:30

On Solaris (and likely other old Unixes)

ggrep -r "$yourtext" $directory

Answered   2023-09-20 20:56:30