Problem with sed

Dacelonid · 07-10-2011 02:23PM #1

Hi,
I have written a bash script that uses sed to convert a CSV file to html table elements. (It is an export from Tournament Director that I publish on our poker website).

The script is as follows

#!/usr/bin/bash
for file in $(ls *.csv)
do
    sed -e "s/,/<\/td><td>/g" $file > $file.tmp
    sed -e "s/\(.*\)$/<tr><td class=\"numeric\">\1<\/td><\/tr>/g" $file.tmp > $file
done
/usr/bin/rm *.tmp

So all it does is scans the directory for .csv files and in each one, it replaces , with </td><td> and then replaces the start and end of the line with some other info.

The script works fine except for 2 small bugs, one serious enough, the other just an annoyance

1) The last line is not output to the file
2) there is a newline character at the end of every line so the last <\td><\tr> is on the next line of the file.

Now issue two is something I can live with, but issue 1 is serious enough as the league table is always missing the last person.

example data

#,First Name,Last Name,Points,Buy-ins,Hits,Final Tables
1,Stan,Smith,303.00,10,13,6
2,Francine,Smith,291.00,10,11,5
3,Hayley,Smith,284.00,10,20,7
4,Steve,Smith,277.00,9,17,3
5,Roger,Smith,255.00,10,5,5
6,Klaus,Smith,254.00,9,16,4

example output

<tr><td class="numeric">#</td><td>First Name</td><td>Last Name</td><td>Points</td><td>Buy-ins</td><td>Hits</td><td>Final Tables
</td></tr>
<tr><td class="numeric">1</td><td>Stan</td><td>Smith</td><td>303.00</td><td>10</td><td>13</td><td>6
</td></tr>
<tr><td class="numeric">2</td><td>Francine</td><td>Smith</td><td>291.00</td><td>10</td><td>11</td><td>5
</td></tr>
<tr><td class="numeric">3</td><td>Hayley</td><td>Smith</td><td>284.00</td><td>10</td><td>20</td><td>7
</td></tr>
<tr><td class="numeric">4</td><td>Steve</td><td>Smith</td><td>277.00</td><td>9</td><td>17</td><td>3
</td></tr>
<tr><td class="numeric">5</td><td>Roger</td><td>Smith</td><td>255.00</td><td>10</td><td>5</td><td>5
</td></tr>

Can anybody spot the obvious problem that I just can't see.
Thanks

Procasinator · 07-10-2011 03:26PM

I just ran your script against your example CSV data and I got this (using Cygwin, mind you):

[HTML]
<tr><td class="numeric">#</td><td>First Name</td><td>Last Name</td><td>Points</td><td>Buy-ins</td><td>Hits</td><td>Final Tables</td></tr>
<tr><td class="numeric">1</td><td>Stan</td><td>Smith</td><td>303.00</td><td>10</td><td>13</td><td>6</td></tr>
<tr><td class="numeric">2</td><td>Francine</td><td>Smith</td><td>291.00</td><td>10</td><td>11</td><td>5</td></tr>
<tr><td class="numeric">3</td><td>Hayley</td><td>Smith</td><td>284.00</td><td>10</td><td>20</td><td>7</td></tr>
<tr><td class="numeric">4</td><td>Steve</td><td>Smith</td><td>277.00</td><td>9</td><td>17</td><td>3</td></tr>
<tr><td class="numeric">5</td><td>Roger</td><td>Smith</td><td>255.00</td><td>10</td><td>5</td><td>5</td></tr>
<tr><td class="numeric">6</td><td>Klaus</td><td>Smith</td><td>254.00</td><td>9</td><td>16</td><td>4</td></tr>
[/HTML]

Which doesn't seem to display either of the issues you mentioned.

Dacelonid · 07-10-2011 03:30PM

Damn,
I am running it on Solaris. Don't know if that makes a difference.
nice to know though that there is nothing obviously wrong with the script. Must try it on some other shell maybe
Cheers

carveone · 07-10-2011 03:44PM

I think the issue is with .*$ which will match the line feed too. Although I cannot see why that would affect it unless there's a bug in Solaris sed (very possible!).

Perhaps matching the [\r\n] at the end would help (I don't have sed on this machine right now so I can't help more!) Plus you can do this all in one line you know!

carveone · 07-10-2011 04:07PM

I'll add that when I used Solaris, sed was old and crufty and I threw in the towel often and used awk.

#!/usr/bin/bash
for file in $(ls *.csv)
do
    awk '{gsub(/,/, "</td><td>");print "<tr><td class=\"numeric\">$0</td></tr>"}' $file > $file.tmp
    mv $file.tmp $file
done

or something... Even ex would do if you can figure it out....

Problem with sed

Comments