Problem with sorting script - two column sort (Solved)

Using applications, configuring, problems
Post Reply
Message
Author
hansol
Posts: 23
Joined: Sat 23 Nov 2013, 01:55

Problem with sorting script - two column sort (Solved)

#1 Post by hansol »

Hi guys,

I'm going nuts fighting with the "sort" command. I can get it to work for a single sort, but having it do a double one seems to be giving me trouble.

I have the following data:

Code: Select all

101  A
101  B
101  C
102  A
102  B
102  C
103  A
103  B
103  C
And I want to sort the data alphabetically, and then by number, so that I wind up with:

Code: Select all

101 A
102 A
103 A
101 B
102 B
103 B
101 C
102 C
103 C
Any help would be appreciated. I've tried a few variations of the following, but with no luck:

Code: Select all

sort -k2,2 -k1n ~/[filename].csv > [sortedfile].csv

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#2 Post by musher0 »

Hello, hansol.

You may want to try:

Code: Select all

sort -k2 -n -o 101-2-3.txt 101-abc.txt
more 101-2-3.txt
where file 101-abc.txt contains your original unsorted data and
file 101-2-3.txt contains the sorted results.

Somehow the "string numerical value" comes into play. (Parameter -n)
Otherwise, the listings towards the bottom, under "103", are wrongly sorted.

Best regards.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

hansol
Posts: 23
Joined: Sat 23 Nov 2013, 01:55

#3 Post by hansol »

Musher,

Thanks very much! That certainly got it sorted out. I'm staring at your code trying to figure out "why" it works, and can't quite grasp it, but all I care about is that it does indeed work! :D

You saved me hours of googling, so thanks again.

User avatar
L18L
Posts: 3479
Joined: Sat 19 Jun 2010, 18:56
Location: www.eussenheim.de/

#4 Post by L18L »

hansol wrote:Musher... saved me hours of googling
Or minutes of

Code: Select all

sort --help

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#5 Post by musher0 »

hansol wrote:Musher,

Thanks very much! That certainly got it sorted out. I'm staring at your code trying to figure out "why" it works, and can't quite grasp it, (..)
Hello, hansol.

My pleasure! I believe that you had to add the -n (string numerical value sort)
because you have a column of numbers as well as a column of letters, even
though you are sorting on the column of letters.

We'd need to validate if this is actually the reason, with various tests:
1) by using your list with the letters first
2) using other sets, such as
ba be 98
bi bo 97
bu by 45
da de 33
di do 56
du dy 11
(or whatever)
and see what happens
or
3) any "real-life" combination of columns with letters and numbers.

I did clean up your list, also, of any unneeded spaces - although technically it
should not have mattered. (You can use the -b parameter if you have only
leading blanks.)

Finally, IMO, these problems are best searched not with google, but through a
meta-searcher such as info.com. Type "linux" in front, too; otherwise you'll get
all the results for all the other OS's as well. Alternately, although it is not a meta-
searcher per se, ask.com finds meaningful results pretty fast (in my experience.)

Have a great day!

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

hansol
Posts: 23
Joined: Sat 23 Nov 2013, 01:55

#6 Post by hansol »

L18L wrote:
hansol wrote:Musher... saved me hours of googling
Or minutes of

Code: Select all

sort --help
Possibly, if I understood "geek-speak" as I call it (a term of endearment in my world.) I'm completely new to the Linux world, so a lot of the --help documents may as well be written in hyroglyphics as I will be the first to admit I don't have the technical know-how to interpret what gets listed. The more I learn the more I am in awe of the people who can understand, interpret, and implement the various linux command line procedures, and can only wish I was at that level.

Thanks to this forum however, I have managed to transition my entire firm from Windows and pay-per-license software to 100% linux and open-source software. It's now cut my annual software costs from likely $10k in outlays to maybe a $250 annual renewal for tax software. Additionally, the various sort codes and functions have now allowed me to completely automate data entry functions, cutting down on user error, as well as increasing the amount of detail in the material produced, all while reducing time involved by about 400%.

So I can't say enough about the users here at the forum. It's because of guys like Musher that I was able to do the above; without little bits of advice and code through-out this transition, I'd still be at the mercy of Bill Gates and his Windows 8 whims.

User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#7 Post by Flash »

hansol wrote:... It's now cut my annual software costs from likely $10k in outlays to maybe a $250 annual renewal for tax software. ..
Good grief! What were you doing?

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#8 Post by musher0 »

Hi, Flash.

It would be interesting to have the details, but I believe hansol off the bat.

I had a SOHO too once. If he has a small business with +/- ten work stations /
employees, including his own, at a cost of +/- 1,000$ per year each, this including
WhineDose software licenses, hardware, Internet connection and the occasional
WhineDose consultant, and maybe add a portion of the hydro bill, that's quite possible.

He may be the frugal type, too! It costs big bucks to run even a small outfit on
WhineDose.

@hansol
Thanks for the thanks! As I said: my pleasure! Happy to help a fellow entrepreneur!

BFN.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

hansol
Posts: 23
Joined: Sat 23 Nov 2013, 01:55

#9 Post by hansol »

Flash,

Musher has it right: small accounting firm with multiple workstations and take-home laptops for staff. Between Windows licenses, accounting software licenses, finance and portfolio tracking software + licenses, tax software licenses, tax reference library software + annual renewals, plus both on and off-site servers, plus backup machines for when the workstations go down, things add up. Then on top of that you have the IT troubleshooting costs for when things aren't working like they should and a third party has to come in and fix things.. Certainly it's just the cost of doing business, with a majority of the cost paid up-front, but it's nice to be able to avoid a lot of that expense entirely now. The tax-specific stuff still requires annual renewals, but most everything else can be open source from what I'm finding.

Making the switch was initially pretty scary, but after trialling the linux and open-source products on just my machine to see how it would work, I'm essentially a convert. Certainly a bit of a learning curve, but the community here has been great in helping overcome the little humps and quirks.

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#10 Post by musher0 »

Here to help! Come back anytime! :)
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

Les Kerf
Posts: 317
Joined: Sun 24 Jun 2012, 13:30

#11 Post by Les Kerf »

hansol wrote:...

Thanks to this forum however, I have managed to transition my entire firm from Windows and pay-per-license software to 100% linux and open-source software...
Just curious as to which distro(s) you are using?

hansol
Posts: 23
Joined: Sat 23 Nov 2013, 01:55

#12 Post by hansol »

Les,

The breakdown of what I'm using is as follows. I'll try and be as comprehensive as possible, but I might forget some things:

* O/S: Puppy Linux 5.7.1 Precise, because it plays nice with the Hamachi VPN software. (I'm too much of a rookie to figure out openVPN.) With the new Slacko 5.7 out, it might work, but I haven't tried it yet.
* LibreOffice as an office suite, combined with old Office 2007 software running WINE.
* Oracle's Virtualbox running as a virtual machine in Puppy, using XP as the O/S. Unfortunately I need the XP environment to run tax software, which won't play nice with WINE.
* Dosbox running proprietary DOS accounting software (This one always gets a second look, but nothing new can top the speed/efficiency of this software, and to write a GUI interface for it would cost a ridiculous amount of money for nothing more than "pretty pictures")
* Gnotime as a time-tracking software.
* Thunderbird as email suite.
* Standard puppy programs like PeasyPDF for PDF creation and manipulation and such.
* Investment software is essentially a string of shell scripts for manipulating and extracting the pertinent information and one-click importing the info into the accounting software. Open source for portfolio management is still scarce in terms of finding a program that can accurately calculate cost-base tracking info. A lot of programs (such as gnucash and money manager X) will track shares, plus the buy/sell cost, but can't impute the capital gain/loss on those transactions.
* Hamachi software for VPN connections when needing to access the on-site server.
* Google cloud for cloud storage.
*Bcrypt with two levels of encryption (encrypting the encrypted file) for uploading the cloud storage information so that Google can't read it. (In theory haha)
* Shell scripts paired with Cron for automatic backups of all information on the machine, and automatic uploads to the server as backup.

Anyway, that basically sums it up. Honestly the biggest killer is having multiple desktop workstations, laptops, and backups for both, all requiring Windows licenses, PLUS the related Microsoft Office suite software, PLUS Adobe Acrobat licenses, that really add up. Again, it's the cost of doing business, but when there's a platform out there that accomplishes all of the above for free, but more importantly does it in a RELIABLE manner, that's the main thing. It's one thing to play around with linux as a hobby, but when you are running your business off it, you need to have it work consistently and without issue. So far I've found that once Puppy is installed and configured to the firm "standard", there aren't any issues at all. I did try slackware, but installation and configuration, as well as installing software I found was very difficult, with a steep learning curve. It can be done (I got a test slackware machine running eventually), but the time involved vs. having Puppy work instantly was a turn-off for me.

Les Kerf
Posts: 317
Joined: Sun 24 Jun 2012, 13:30

#13 Post by Les Kerf »

Thanks for the feedback, it was very informative. Sure is nice to know what can done with Puppy :-)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#14 Post by musher0 »

Many, many thanks, hansol.

You have just provided this community with a very good, inspiring and eloquent
evidence that Puppy can be more than a hobby system, it can be used as a
professional system too.


And thank to Les_Keef for asking the question that prompted your report.

We need more reports like this -- to spread the idea among the general public -- and
even among our own developers... -- that PuppyLinux can be an excellent basis for
serious business and professional uses.

I find hansol's report very interesting because, on the French side of the forum, with the
help and contributions of other Puppy forum members, I've initiated, since Dec. 2012,
the thread "Puppy for micro-business: thesaurus of resources" (translated title)
http://www.murga-linux.com/puppy/viewtopic.php?t=82649
There you will find reviewed or simply mentioned a lot of templates and applications
that can be used for SOHO's.

hansol, can I have your permission to translate your report above and post the
translation in our French thread? I believe that such serious uses of Puppy need
to be more widely known.

Thanks in advance and best regards.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#15 Post by musher0 »

Hello, all.

Anyone wanting to use PuppyLinux for work may also find this interesting.

I got back in touch with e_mattis, who has produced a Small Business Pup
based on PuppyLucid 5.2.8, about 16 months ago.

Quoting his reply from yesterday:
Hey Christian,
This is Eric (e_mattis). Thanks for getting back in touch with me! I hope the SBP is working out to your expectations. The direct link to the download ishttp://data.deadaccurate.com/SBPiso.tar.gz. The link to the discussion is . Puppy Linux Discussion Forum :: View topic - Francisation du Small Business Pup d'e_mattis

(forum illustration removed)

I haven't had time to do anything else to the project - and it doesn't look as if I will anytime soon. you might also want to look at http://data.deadaccurate.com/SBPp_5.4.3.iso. it was supposed to be the second such distro on a different platform and with some different apps. but fair warning - it will probably be buggy!

thanks again and let me know if you need anything!

Your friend,
Eric
I can say that Eric's Lupu 5.2.8 derivative is excellent for anyone needing
a Point of Sale software. It's a treat for the eye as well, and it has other
pertinent SOHO software, too.

I was supposed to fiddle with his second version, Shame on me, I haven't had the time...

Hoping these URLS will be of use to someone. BFN.

Chris (aka musher0)
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
ally
Posts: 1957
Joined: Sat 19 May 2012, 19:29
Location: lincoln, uk
Contact:

#16 Post by ally »


User avatar
Flash
Official Dog Handler
Posts: 13071
Joined: Wed 04 May 2005, 16:04
Location: Arizona USA

#17 Post by Flash »

Hansol and musher0, you two ought to collaborate on a thread about an Enterprise Puppy in the How-to section of the forum. :)

musher0
Posts: 14629
Joined: Mon 05 Jan 2009, 00:54
Location: Gatineau (Qc), Canada

#18 Post by musher0 »

Flash wrote:Hansol and musher0, you two ought to collaborate on a thread about an Enterprise Puppy in the How-to section of the forum. :)
Thanks for the thought, Flash.

Indeed, it would certainly be very good to have hansol and his expertise on board.
For the last couple of days, Eric (e_mattis) and I have started to exchange seed ideas
about a new SOHO Puppy, through PM.

Maybe considering an "Enterprise Puppy" is a bit ambitious at this point, but with
like-minded people working together in some sort of loose team structure, who knows,
anything's possible.

BFN.

musher0
musher0
~~~~~~~~~~
"You want it darker? We kill the flame." (L. Cohen)

User avatar
Karl Godt
Posts: 4199
Joined: Sun 20 Jun 2010, 13:52
Location: Kiel,Germany

#19 Post by Karl Godt »

hansol wrote:Musher,

Thanks very much! That certainly got it sorted out. I'm staring at your code trying to figure out "why" it works, and can't quite grasp it, but all I care about is that it does indeed work! :D

You saved me hours of googling, so thanks again.
First :
Your code had two times the -k switch
The -k switch is commonly used together with the -t switch :

Code: Select all

LIST="101  A
101  B
101  C
102  A
102  B
102  C
103  A
103  B
103  C"

sortDELIMITER=" "

echo "$LIST" | sort -k2 -t"$sortDELIMITER"
101  A
102  A
103  A
101  B
102  B
103  B
101  C
102  C
103  C
And be aware that Puppies might use the busybox sort applet instead of the binary :

Code: Select all

LIST="101  A                              
101  b
101  C
102  A
102  b
102  C
103  a
103  B
103  c"

echo "$LIST" | sort -k2 -t"$sortDELIMITER"
103  a
101  A
102  A
101  b
102  b
103  B
103  c
101  C
102  C

echo "$LIST" | busybox sort -k2 -t"$sortDELIMITER"
101  A
102  A
103  B
101  C
102  C
103  a
101  b
102  b
103  c

disciple
Posts: 6984
Joined: Sun 21 May 2006, 01:46
Location: Auckland, New Zealand

#20 Post by disciple »

Interesting thread.
Hansol, you might also want to check out pdfshuffler and pdfsam if you haven't already. And one day you might find Kingsoft Office or perhaps Softmaker Office saves you from Wine :)
Do you know a good gtkdialog program? Please post a link here

Classic Puppy quotes

ROOT FOREVER
GTK2 FOREVER

Post Reply