How to Remove CTRL-M characters From a File in UNIX and Linux? Example

I wanted to transfer some files from Windows to Unix using FileZilla, but the problem arises when these files are transferred (Ascii or Binary mode both) and opened using VI we get ^M characters, also known as CTRL-M characters. I searched about this but the solutions were to remove these ^M characters when files are transferred using utilities. Is there any way that these ^M characters do not appear in the first place? Well, my search continues but I will share the solution which worked for me for removing control M characters i.e. CTRL-M or ^M characters.

There are several UNIX commands like dos2unix which can be used to convert Windows or DOS generated files to UNIX one. You can also use sed command (stream editor) to remove CTRL-M characters without opening the file, very useful if you are removing CTRL-M characters from a large file. Alternatively, you can use the VI command to open the file and replace ^M characters with nothing.

Unix uses a single Line Feed (LF) character as the line break. Windows/DOS uses 2 characters: Carriage Return/Line Feed (CR/LF). The control M characters appear in a file when you transfer them from Windows to UNIX (see How Linux works).

So the answer is no, you cannot eliminate the CR\LF (unless you write all your text in one line) but you can still remove those CTRL-M characters by using one of the two mentioned ways:

By the way, if you are new to Linux then I also suggest you go through a comprehensive Linux course to learn some basics commands and fundamentals like Linux file system, permissions, and other basic things.

If you need an online course, I highly recommend Linux Mastery: Master the Linux Command Line in 11.5 Hours on Udemy. It’s a very practical and hands-on course to learn Linux fundamentals in a quick time. It’s also very affordable and you can buy in just $10 on Udemy flash sales which happen every now and then.

2 ways to remove Control-M characters from a file in UNIX

Here is a couple of ways to delete Control-M characters from a file, in the first way we try to use a command which converts a DOS file to UNIX format and in the second way we literally remove Control-M characters using UNIX commands e.g. sed or vi

1st Way 

$ dos2unix abc.txt

As the name suggests the dos2unix command converts a DOS or Windows generated file to UNIX file i.e. it replaces all CRLF to LF. See, The Linux Command Line: A Complete Introduction to learning more about the dos2unix command.

2nd Way

You may need to do this when you import a text file from MS-DOS or (Window 8, 8.1, 10 or any other version), and forget to transfer it in ASCII or text mode. Here are a couple ways to do it, you can choose the one you like.

The easiest way is probably to use the stream editor sed to remove the ^M characters as shown in the following example

$ sed -e "s/^M//" filename > newfilename

To enter ^M, type CTRL-V, then CTRL-M. I mean, hold down the CTRL key then press V and M in succession. The sed command is really helpful in case of removing ^M characters from large and huge files because you don’t need to open the files into VI or any other editor.

Btw, here is a screenshot of a how Control-M character looks like inside a file in UNIX, I am sure you have seen it before:

Btw, you can also remove ^M characters by opening the file in VI editor and replacing them with nothing as shown in the following command:

$ vi filename
Inside vi [in ESC mode] type: :%s/^M//g

To enter ^M, type CTRL-V, then CTRL-M i.e. hold down the CTRL key then press V and M in succession.

If you are an Emacs fan then don’t get disappointed because you can also remove CTRL M characters in Emacs by following the below steps:

Go to the beginning of the document
Type: M-x replace-string RET C-q C-m RET RET
where “RET” means and C-q and C-m mean .

See Practical Guide to Linux Commands, Editors, and Shell Programming 3rd Edition by Mark G. Sobell to learn more about UNIX commands and editors.

In short, control-m characters will get appended to a file when a file is transferred from windows to a UNIX machine. There are four ways it can be removed.

Using vi editor:

:%s/^M//g

Using col command:

$ cat filename | col -b > newfilename

Using sed command:

$ sed 's/^M//g' filename > newfilename

Using dos2unix command:

$ dos2unix filename newfilename

Now, let’s see a complete example of removing control m characters from a file. You can see below we have a file with ^M characters, after using the sed command those will be removed and verified again by using the cat command.

$ cat -v jsk
hi^M
jsk^M
$ cat jsk | col -b > jsk.new
$ cat -v jsk.new
hi
jsk
$ sed 's/^M//g' jsk > jsk.new2
$ cat -v jsk.new2
hi
jsk

Note: Remember how to type control M characters in UNIX, just hold the control key and then press v and m to get the control-m character.