Puppy Linux Discussion Forum

musher0 · Last edited by musher0 on Sat 12 Jan 2013, 10:14; edited 7 times in total

Update, Sat., Jan. 12, 2013.

Hello.

A year almost to the day, here is csvfix 1.50. The pet archive for this version is basically structured the same, except that csvfix and the docs have been updated to the latest version, v. 1.50.

You can see what has changed here: http://code.google.com/p/csvfix/wiki/ChangeLog. In particular, the new

musher0

Hello again.

The following will speak volumes, to whoever got headaches trying to successfully process a csv database, about the usefulness of this program. It also describes what csvfix command to use to solve a particular problem. From the docs.

TWYL.

musher0

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Solutions to Common Problems
Introduction ››

Here are some of the common problems that you may experience when dealing with CSV data, and brief suggestions as to how CSVfix can help you with them:

"The fields in the data are in the wrong order"
* Reorder them using the order command.

"The records in the data are in the wrong order"
* Sort them using the sort command. Remove duplicates with the unique command.

"Some fields are missing from my data"
* If you simply need to add empty fields to keep your application happy, use the pad command. If you need to enrich your data with values from another file, use the join command. If you need to add a fixed string to the data, use the put command.

"My CSV data isn't comma-separated!"
* Use the -sep and -rsep flags to tell CSVfix what separator to use.

"I need to read tab-separated data"
* Use the read_dsv command with the -s '\t' flag to convert tab-separated data to CSV.

"I need to convert XML to CSV"
* The from_xml command probably does what you need.

"I need to perform calculations on some of the fields in my CSV data"
* The eval command provides arithmetic and other functions you can apply to your CSV data.

"I need to extract data from the middle of a field"
* Use the split_fixed and split_char commands to break fields up into sub fields. If you need to build a field from values in other fields, use the merge command.

"I need to remove some lines from a CSV file before processing it further"
* The find and remove commands allow you to filter your CSV files using regular expressions, value ranges and field lengths. You can also create exclusion lists and use the join command with the -inv flag to exclude selected rows.

"I need to merge the data from two CSV files"
* Use the order command to get the fields in the same order, and then the unique command to merge them, discarding duplicates.

"I want to add record numbers to my CSV data"
* The sequence command provides flexible record numbering.

"I want to split my data into different files"
* The file_split command can split CSV data streams depending on field values.

"My data is full of duplicate values"
* Remove duplicates using the unique command.

"The data I get given is always full of invalid values"
* Validate your data using the validate command. Remove bad values with the remove command.

"I need to convert CSV data into XML"
* The to_xml command does exactly this.

"I need to convert CSV data to this weird format"
* Use write_fixed, write_dsv and the printf and template commands to format your output.

"I want to extract data from a SQL database as CSV"
* In the Windows version of CSVfix, the odbc_get command allows you to extract data from SQL databases.

"I need to import data into a database, but the data is in a weird format"
* Use CSVfix to convert the data to CSV using the read_fixed, read_dsv, and/ot read_multi commands, and then convert the CSV to SQL INSERT statements using sql_insert.

"I need special date formatting"
* Use the date_format and date_iso comnands to read and re-format dates.

Copyright © 2011 Neil Butterworth

~~~~~~~~~~~~~~~~~~~~~~~~~~~~

gcmartin · Joined: 14 Oct 2005 Posts: 3440 Location: Earth

Thanks @Musher0

I seem to remember few years ago a similar Windows Program for creating and processing CSVs. This was very useful for prepping data for DB entry.

Does CSVFIX has a front-ent gui for processing CSVs or is it a CLI only tool.

Thanks in advance

musher0 · Posted: Fri 13 Jan 2012, 01:40 Post subject:

Hi, gcmartin.

No csvfix is only CLI. But I "intuited" Smile

your question, as I came back to upload dmcsv.

dmcsv is a simple csv viewer -- don't expect CSVed for Linux, here --, but still it will help you view the results of your processing with csvfix -- if you're unsure in your mind of what the process will do. It is less abstract of course than viewing your modified csv file with less -- which is fine, too.

In any case, enjoy! And have a great day!

P.S. Here is a very short description of dmcsv, as given by the author:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
DMcsvEditor is simple csv/tab files editor
darhmedia.hu [where you'll find the archive of the file]
darhmedia@gmail.com
goo.gl/77BAE
*** this version developing on Trisquel GNU/Linux ***
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

And a screenshot.

musher0 · Posted: Fri 13 Jan 2012, 02:50 Post subject:

Here is an easy way to integrate the two.

And at the same it will give the opportunity to explain some csvfix commands.

Let's start with the csvfix commands.

For example, let's say you wish to number a list of clients, putting the entry number as the first field.

You would thus use the command

musher0 · Posted: Fri 13 Jan 2012, 04:32 Post subject:

If it's of any help, the latest version of less is here:

http://murga-linux.com/puppy/viewtopic.php?search_id=200079825&t=75161

TWYL.

musher0 · Posted: Sat 12 Jan 2013, 09:42 Post subject:

Hi.

Here is the complete list of commands for csvfix, v. 1.50.

amigo · Joined: 02 Apr 2007 Posts: 2133

Where did you find dmcsv at?

Distracting a slight bit from the original subject, I recently ram across another light-weight database called 'recutils'. It's a CLI database which lends itself well to scripting or front-ending with some dialog program. Also, it uses flat plain-text files so the databse can easily be worked with using grep, awk, sed or any text editor you like.
recutils is in the ubuntu repos so you can look for it there.

My search for dmcsv came up empty.

musher0 · Posted: Sat 19 Jan 2013, 15:53 Post subject:

Hello, amigo.

Thanks for enquiring. These are the search results I got with ask:
http://www.ask.com/web?q=dmcsv%20csv&o=101451&l=dis
I had to insist with the search parameters : "dmcsv csv".

The Linux version is apparently here :
http://linux.softpedia.com/get/Text-Editing-Processing/Others/Text-Editor/DMcsvEditor-56283.shtml

However, that may be an error, since the latest available Linux version listed on the author's site appears to be v. 0.6... Please see : http://code.google.com/p/dmcsveditor/downloads/list?q=label:Latest

(Linux version 0.6 of dmcsv is provided above in this thread.)

dmcsv obviously exists for Windows :
http://www.softpedia.com/get/Programming/File-Editors/DMcsvEditor.shtml

and even as a portable app:
http://www.portablefreeware.com/?id=2183

Best regards.

musher0

musher0 · Posted: Sat 19 Jan 2013, 16:04 Post subject:

To download recutils : http://archive.ubuntu.com/ubuntu/pool/universe/r/recutils/recutils_1.5-1_i386.deb

I'll be testing it shortly. Thanks again for the tip, amigo.

musher0

amigo · Joined: 02 Apr 2007 Posts: 2133

Eeeww! dmcsv=Lazarus/Delphi

Still, since csvfix is all CLI, it would also be possible to make a frontend using gtkdialog or other such animal.

musher0 · Posted: Sat 19 Jan 2013, 16:40 Post subject:

amigo · Joined: 02 Apr 2007 Posts: 2133

Oh, I'm definitely suggesting! At least until I have a need or an itch to satisfy somebody else's need. Come on, there's lots of gkdialog programmers around here -even though I rather see a regular gtk GUI or something done with something beside gtkdialog... Far be it from me to implement something which others could do instead... I have my hands full already. But you can (nearly) always count on me for suggestions and wry criticism!

Pelo

Many thanks, Musher0
You are improving our knowledge a lot !
See you soon.
Dom Pelo de la casbah francesa.

Pelo

Many thanks, Musher0
You are improving our knowledge a lot !
See you soon.
Dom Pelo de la casbah francesa.