Skip to main content

Replace String in Files/File


1. Replacing all occurrences of one string with another in all files in the current directory:
These are for cases where you know that the directory contains only regular files and that you want to process all non-hidden files. If that is not the case, use the approaches in 2.
All sed solutions in this answer assume GNU sed. If using FreeBSD or OS/X, replace -i with -i ''.
·         Non recursive, files in this directory only:
·         sed -i -- 's/foo/bar/g' *
perl -Ti -pe 's/foo/bar/g' ./*
·         Recursive, regular files (including hidden ones) in this and all subdirectories
find . -type f -exec sed -i 's/foo/bar/g' {} +
If you are using zsh:
sed -i -- 's/foo/bar/g' **/*(D.)
(may fail if the list is too big, see zargs to work around).
If you are using bash, bash having no support for glob qualifiers, you can't check for regular files:
shopt -s globstar
shopt -s dotglob
Then:
sed -i -- 's/foo/bar/g' **/*
2. Replace only if the file name matches another string / has a specific extension / is of a certain type etc:
·         Non-recursive, files in this directory only:
·         sed -i -- 's/foo/bar/g' *baz*    ## all files whose name contains baz
sed -i -- 's/foo/bar/g' *.baz    ## files ending in .baz
·         Recursive, regular files in this and all subdirectories
find . -type f -name "*baz*" -exec sed -i 's/foo/bar/g' {} +
If you are using bash:
shopt -s globstar
shopt -s dotglob
Then:
sed -i -- 's/foo/bar/g' **/*baz*
sed -i -- 's/foo/bar/g' **/*.baz
If you are using zsh:
sed -i -- 's/foo/bar/g' **/*baz*(D.)
sed -i -- 's/foo/bar/g' **/*.baz(D.)
The -- serves to tell sed that no more flags will be given in the command line. This is useful to protect against file names starting with -.
·         If a file is of a certain type, for example, executable (see man find for more options):
find . -type f -executable -exec sed -i 's/foo/bar/g' {} +
zsh:
sed -i -- 's/foo/bar/g' **/*(D*)
3. Replace only if the string is found in a certain context
·         Replace foo with bar only there is a baz later on the same line:
sed -i ':1;s/foo\(.*baz\)/bar\1/;t1' file
In sed, using \( \) saves whatever is in the parentheses and you can then access it with \1. There are many variations of this theme, to learn more about such regular expressions, see here. We need to repeat the operation for all foo occurrences, which is done with the t conditional branching.
·         Replace foo with bar only if foo is found on the 3d column (field) of the input file (assuming whitespace-separated fields):
gawk -i inplace 'gsub(/foo/,"baz",$3)' file
(need gawk 4.1.0 or newer).
For a different field just use $N where N is the number of the field of interest. For a different field separator (: in this example) use:
gawk -i inplace -F':' 'gsub(/foo/,"baz",$3)' file
Another solution using perl:
perl -i -ane '$F[2]=~s/foo/baz/g; $" = " "; print "@F\n"' foo
NOTE: both the awk and perl solutions will print space separated fields even if the input file had tabs. For a different field use $F[N-1] where N is the field umber you want and for a different field separator use (the $"=":" sets the output field separator to :):
perl -i -F':' -ane '$F[2]=~s/foo/baz/g; $"=":";print "@F"' foo
·         Replace foo with bar only on the 4th line:
·         sed -i '4s/foo/bar/g' file
·         gawk -i inplace 'NR==4{gsub(/foo/,"baz")};1' file
perl -i -pe 's/foo/bar/g if $.==4' file
4. Multiple replace operations: replace with different strings
You can combine sed commands:
sed -i 's/foo/bar/g; s/baz/zab/g; s/Alice/Joan/g' file
or Perl commands
perl -i -pe 's/foo/bar/g; s/baz/zab/g; s/Alice/Joan/g' file
If you have a large number of patterns, it is easier to save your patterns and their replacements in a sed script file:
#! /usr/bin/sed -i
s/foo/bar/g
s/baz/zab/g
Or, if you have too many patterns pairs for the above to be feasible, you can read pattern pairs from a file (two space separated patterns, $pattern and $replacement, per line):
while read -r pattern replacement; do  
   sed -i "s/$pattern/$replacement/" file
done < patterns.txt
That will be quite slow for long lists of patterns and large data files so you might want to read the patterns and create a sed script from them instead:
sed -f <(awk '{printf "s/%s/%s/g\n", $1, $2}' patterns.txt) -i -- file.txt
Then, run the sed script on your input file(s):
sed -f sedscript.txt inputfile.txt
5. Multiple replace operations: replace multiple patterns with the same string
Replace any of foobar or baz with foobar
sed -Ei 's/foo|bar|baz/foobar/g' file
or
perl -i -pe 's/foo|bar|baz/foobar/g' file


Comments

Popular posts from this blog

Zabbix History Table Clean Up

Zabbix history table gets really big, and if you are in a situation where you want to clean it up. Then we can do so, using the below steps. Stop zabbix server. Take table backup - just in case. Create a temporary table. Update the temporary table with data required, upto a specific date using epoch . Move old table to a different table name. Move updated (new temporary) table to original table which needs to be cleaned-up. Drop the old table. (Optional) Restart Zabbix Since this is not offical procedure, but it has worked for me so use it at your own risk. Here is another post which will help is reducing the size of history tables - http://zabbixzone.com/zabbix/history-and-trends/ Zabbix Version : Zabbix v2.4 Make sure MySql 5.1 is set with InnoDB as innodb_file_per_table=ON Step 1 Stop the Zabbix server sudo service zabbix-server stop Script. echo "------------------------------------------" echo " 1. Stopping Zabbix Server ...

Installing Zabbix Version 2.4 Offline (Zabbix Server without Internet).

There might be situations where you have a remote/zabbix server which does not have internet connectivity, due to security or other reasons. So we create a custom repo on the remote/zabbix server so that we can install zabbix using rpms Here is how we are planning to do this. Download all the dependency rpms on a machine which has internet connection, using yum-downloadonly or repotrack . Transfer all the rpms to the remote server. Create a repo on the remote server. Update yum configuration. Install. NOTE: This method can be used to install any application, but here we have used zabbix as we had this requirement for a zabbix server. Download dependent rpms . On a machine which has internet connection install the package below. And download all the rpms . Make sure the system are similar (not required to be identical - At-least the OS should be of same version) mkdir /zabbix_rpms yum install yum-downloadonly Downloading all the rpms to location /zabbix_rpms/ ,...

Access Filter in SSSD `ldap_access_filter` [SSSD Access denied / Permission denied ]

Access Filter Setup with SSSD ldap_access_filter (string) If using access_provider = ldap , this option is mandatory. It specifies an LDAP search filter criteria that must be met for the user to be granted access on this host. If access_provider = ldap and this option is not set, it will result in all users being denied access. Use access_provider = allow to change this default behaviour. Example: access_provider = ldap ldap_access_filter = memberOf=cn=allowed_user_groups,ou=Groups,dc=example,dc=com Prerequisites yum install sssd Single LDAP Group Under domain/default in /etc/sssd/sssd.conf add: access_provider = ldap ldap_access_filter = memberOf=cn=Group Name,ou=Groups,dc=example,dc=com Multiple LDAP Groups Under domain/default in /etc/sssd/sssd.conf add: access_provider = ldap ldap_access_filter = (|(memberOf=cn=System Adminstrators,ou=Groups,dc=example,dc=com)(memberOf=cn=Database Users,ou=Groups,dc=example,dc=com)) ldap_access_filter accepts standa...