The FreeBSD Diary

The FreeBSD Diary (TM)

Providing practical examples since 1998

If you buy from Amazon USA, please support us by using this link.
[ HOME | TOPICS | INDEX | WEB RESOURCES | BOOKS | CONTRIBUTE | SEARCH | FEEDBACK | FAQ | FORUMS ]
FreeBSD Support
 New Topic  |  Go to Top  |  Go to Topic  |  Search  |  Log In   Newer Topic  |  Older Topic 
 how to get rid of ^M in textfiles?
Author: g2k 
Date:   13-08-01 18:28

i know there are tools for this out there, but i dont know any by name. so can anyone tell me a program to get rid of those ^M at the end of each line in a textfile?

^Ms get there when textfiles are created with any windows texteditors..not that i use them, but i currently have to work with a lot of these files, around 77Megs.

i'd use perl to get rid of them, but unfortunately pattern matching ^M (s/^M+$//g;) wont work for some reason..

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Erin 
Date:   13-08-01 18:37

I use;

tr -d '\015' < in-file > out-file

I have used this for a while with no problems. The ^M come from FTPing a text file is binary mode. If you transfer it in ASCII you will not get those. :-)


Erin

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Erin 
Date:   13-08-01 18:40

g2k wrote:
>
> i'd use perl to get rid of them, but unfortunately pattern
> matching ^M (s/^M+$//g;) wont work for some reason..

The reason is you are trying to match two characters in "^M" where the machine only see's one. If you matched the ASCII code for it (and I don't remember what it is) it would work fine.


Erin

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Ryan Masse 
Date:   13-08-01 18:46

using s/^M//g works for me

ryan

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: g2k 
Date:   13-08-01 19:00

yeah, i've noticed that, but neither do i know the ascii code for ^M.

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: g2k 
Date:   13-08-01 19:05

> I have used this for a while with no problems. The ^M come
> from FTPing a text file is binary mode. If you transfer it in
> ASCII you will not get those. :-)

cool, i didn't know that ;)
but besides that, editing such files in wordpad adds the ^M character too, doesn't it? i think so.

i'll try tr -d '\015', thanks.

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Dan Langille 
Date:   13-08-01 20:47

Did any of you think to use <A HREF="/search.php">search</A> and look for "control M"?

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Dan Langille 
Date:   13-08-01 20:54

Actually, this thread is so interesting, I'm tempted to copy it over to feedback on the CONTROL-M article itself.

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Jay 
Date:   14-08-01 15:40

I'm a big fan of dos2unix (use the google, the force is weak), it works well. unix2dos adds the ^M's back in. Good stuff.

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: el_kab0ng 
Date:   14-08-01 17:41

If the file was manageable, you could simply download it back to a windows box, then re-upload it back as ASCII. That removes all the binary ^M characters (at least using WS_FTPLE). Notepad itself does NOT add these characters, but Word and Wordpad do because they add formatting. Even still, uploading these in ASCII should remove the ^M's.

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Jay 
Date:   14-08-01 18:09

IIRC, the ^M is the second character for CRLF, which DOS uses. *nix only uses one of either CR or LF (carriage return, line feed). Or maybe not... For all I know, binary uses both and ascii uses one. I'm not going to be writing a text editor anytime soon, so its not too much of a concern (reinventing the wheel is a waste of my time, I'm an admin, not a serious developer).

I just use the tools I have to convert between the two (dos2unix). It's rather quick, takes about 2 seconds to convert every file in the /www dir on my webserver (140+ files).

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Daniel Schrock 
Date:   14-08-01 22:06

This is not a result of ftp'ing an ascii file in binary mode.

This is the result of text editing in Windows with an inferior editor(notepad/wordpad).
You should acquire a better notepad/wordpad replacement that lets you save in unix format.

Or better yet...use vi.

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: el_kab0ng 
Date:   14-08-01 22:53

notepad doesn't add ^M characters. Wordpad and Word might.

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Don 
Date:   16-08-01 01:38

perl -p -i -e 's/\r\n/\n/s' file name or wildcard

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Alan Fox 
Date:   23-02-02 19:50

Based on the last (good, working) perl script, I wrote a quick shell script that does all this for me.

The contents of the shell script are as follows:

#!/bin/sh
perl -p -i -e 's/\r\n/\n/s' $*

I named this script "dos2unix", placed it in the /usr/local/bin directory, and ran a quick "chmod 755 dos2unix" to get it working.

Just type "dos2unix file1.txt file2.txt ...", and it happily does its stuff.

The filename is optional. The name dos2unix collides with the name of another handy utility, also mentioned in this forum topic. I only named it "dos2unix" because I am used to dealing with that utility on Solaris systems, and I would not have to memorize any extra tool names. Note, by the way, that the usage of this tool differs from the original dos2unix.

Reply To This Message
 
 Re: how to get rid of ^M in textfiles?
Author: Dan Langille 
Date:   15-01-03 02:22

See also http://www.freebsddiary.org/phorum/list.php?f=3&article_id=308

Reply To This Message
 Forum List  |  Threaded View   Newer Topic  |  Older Topic 


 Forum List  |  Need a Login? Register Here 
 User Login
 User Name:
 Password:
 Remember my login:
   
 Forgot Your Password?
Enter your email address or user name below and a new password will be sent to the email address associated with your profile.
How to get the most out of the forum

phorum.org