Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Fri 17 Aug 2018, 07:03
All times are UTC - 4
 Forum index » Off-Topic Area » Programming
How to find two consecutive lines matching two SOLVED
Post new topic   Reply to topic View previous topic :: View next topic
Page 1 of 2 [21 Posts]   Goto page: 1, 2 Next
Author Message
stemsee

Joined: 27 Jun 2013
Posts: 2260
Location: In The Way

PostPosted: Mon 09 Apr 2018, 16:14    Post subject:  How to find two consecutive lines matching two SOLVED
Subject description: then insert a line inbetween them?
 

So, the problem is scanning for wifi APs and finding APs that have no ESSID:"" line...I suspect these may be security cameras or some such devices. For my wifi app there has to be an ESSID line so the gui can correctly display found APs, one missing ESSID and all hell breaks loose!

Here are the raw scan results the first cell is missing the ESSID:"" line, but there could be other cells missing it too ... how to search each cell body and insert this ESSID:"" line if Cell has no ESSID line???

i looked for solutions but none worked correctly. They could insert the line either after a single line match, or before but not match two consecutive lines and insert between them!!!
Code:

          Cell 01 - Address: 2E:FE:4A:5A:48:1B
                    Channel:13
                    Frequency:2.472 GHz (Channel 13)
                    Quality=49/70  Signal level=-61 dBm 
                    Encryption key:on
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s
                              9 Mb/s; 12 Mb/s; 18 Mb/s
                    Bit Rates:24 Mb/s; 36 Mb/s; 48 Mb/s; 54 Mb/s
                    Mode:Master
                    Extra:tsf=0000001619acbc26
                    Extra: Last beacon: 64ms ago
                    IE: Unknown: 0006646F6F676565
                    IE: Unknown: 010882848B960C121824
                    IE: Unknown: 03010D
                    IE: Unknown: 32043048606C
                    IE: Unknown: 2A0100
                    IE: Unknown: 2D1A300103FF00000000000000000000000100000000000000000000
                    IE: Unknown: 3D160D000000000000000000000000000000000000000000
                    IE: Unknown: 7F0100
                    IE: Unknown: DD180050F2020101800003A4000027A4000042435E0062322F00
                    IE: IEEE 802.11i/WPA2 Version 1
                        Group Cipher : CCMP
                        Pairwise Ciphers (1) : CCMP
                        Authentication Suites (1) : PSK
                    IE: Unknown: DD180050F204104A00011010440001021049000600372A000120
                    IE: Unknown: DD07000CE700000000
          Cell 02 - Address: 90:21:06:24:35:71
                    Channel:1
                    Frequency:2.412 GHz (Channel 1)
                    Quality=31/70  Signal level=-79 dBm 
                    Encryption key:on
                    ESSID:"NOWTV65791"
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 18 Mb/s
                              24 Mb/s; 36 Mb/s; 54 Mb/s
                    Bit Rates:6 Mb/s; 9 Mb/s; 12 Mb/s; 48 Mb/s
                    Mode:Master
                    Extra:tsf=00000d2153eede40
                    Extra: Last beacon: 1043ms ago
                    IE: Unknown: 000A4E4F5754563635373931
                    IE: Unknown: 010882848B962430486C
                    IE: Unknown: 030101
                    IE: Unknown: 0706444520010D14
                    IE: Unknown: 200100
                    IE: Unknown: 23021100
                    IE: Unknown: 2A0100
                    IE: Unknown: 2F0100
                    IE: IEEE 802.11i/WPA2 Version 1
                        Group Cipher : CCMP
                        Pairwise Ciphers (1) : CCMP
                        Authentication Suites (1) : PSK
                    IE: Unknown: 32040C121860
                    IE: Unknown: 0B050300060000
                    IE: Unknown: 2D1ABC181BFFFF000000000000000000000000000000000000000000
                    IE: Unknown: 3D1601080400000000000000000000000000000000000000
                    IE: Unknown: 7F080400080000000040
                    IE: Unknown: DD800050F204104A00011010
                    IE: Unknown: DD090010180203000C0000
                    IE: Unknown: DD180050F2020101840003A4000027A4000042435E0062322F00
                    IE: Unknown: 46057200010000
          Cell 03 - Address: 70:50:AF:A9:23:92
                    Channel:6
                    Frequency:2.437 GHz (Channel 6)
                    Quality=20/70  Signal level=-90 dBm 
                    Encryption key:on
                    ESSID:"SKY0D953"
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 18 Mb/s
                              24 Mb/s; 36 Mb/s; 54 Mb/s
                    Bit Rates:6 Mb/s; 9 Mb/s; 12 Mb/s; 48 Mb/s
                    Mode:Master
                    Extra:tsf=000001f04d6956fc
                    Extra: Last beacon: 681ms ago
                    IE: Unknown: 0008534B593044393533
                    IE: Unknown: 010882848B962430486C
                    IE: Unknown: 030106
                    IE: Unknown: 2A0100
                    IE: Unknown: 2F0100
                    IE: IEEE 802.11i/WPA2 Version 1
                        Group Cipher : CCMP
                        Pairwise Ciphers (1) : CCMP
                        Authentication Suites (1) : PSK
                    IE: Unknown: 32040C121860
                    IE: Unknown: 0B050000060000
                    IE: Unknown: 2D1ABC191BFFFF000000000000000000000000000000000000000000
                    IE: Unknown: 3D1606080000000000000000000000000000000000000000
                    IE: Unknown: 7F080400080000000040
                    IE: Unknown: DDA60050F204104A000110104400010
                    IE: Unknown: DD090010180200000C0000
                    IE: Unknown: DD180050F2020101800003A4000027A4000042435E0062322F00
                    IE: Unknown: 46057200010000



the lines end with LF (linux line finish)

Last edited by stemsee on Tue 10 Apr 2018, 20:30; edited 2 times in total
Back to top
View user's profile Send private message MSN Messenger 
musher0


Joined: 04 Jan 2009
Posts: 12569
Location: Gatineau (Qc), Canada

PostPosted: Mon 09 Apr 2018, 16:58    Post subject:  

Hi stemsee.

I do not know a thing about wifi, but this looks to me no different from
a classic text file problem, where you compare lines with a view to insert
(or remove) some separator between identical ones, like when you use a
bash script to edit a software manual in txt format.

First let's suppose we have a try-out.lst file that has:
Quote:
baba
baba

Then we can apply something like this to it:
Code:
> schmurf.lst # we create the receiving file
Insert="blabla"
while read line1;do # we read line 1
    read line2 # we read line 2 # it is little known, but the while...done loop
# can do this.
    [ "$line2" = "$line1" ] && echo -e "$line1\n$Insert\n$line2" >> schmurf.lst || echo -e "$line1\n$line2" >> schmurf.lst
# if both lines are identical, we insert "blabla" between them, else we echo
# them to the receiving text file without change.
    done <try-out.lst # reads from the input text file.

The result should be:
Quote:
baba
blabla
baba
When finished, we can move schmurf.lst to try-out.lst.

It does not matter if the string to insert is < ESSID:"" >, < blabla >, a
space, a line feed, etc. Just change < baba > and < blabla > to what you
need in the code above.

IHTH. If you have questions about this, please ask.
~~~~~~~~~~~
Note --
As a string in a bash script, your < ESSID:"" > will have to be written
Code:
Insert="ESSID:\"\""
to be reproduced as you need it.
_________________
musher0
~~~~~~~~~~
Siempre será canción nueva... (V. Jara, Manifiesto)
Back to top
View user's profile Send private message 
stemsee

Joined: 27 Jun 2013
Posts: 2260
Location: In The Way

PostPosted: Mon 09 Apr 2018, 17:10    Post subject:  

The lines to find are
Code:
                    Encryption key:on
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s

but could also be
Code:
                    Encryption key:off
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s

on/off is unknown

Then insert
Code:
                    ESSID:""

Giving
Code:
                    Encryption key:off
                    ESSID:""
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s

including tabs!

This may happen N number of times in a 30+ AP scan!
Something like this
Code:
REPLACELINE1='                    Encryption key:on'
REPLACELINE2='                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s'
REPLACEWITHLINE2='                    ESSID:""'
sed -n "1h; 1!H; \${ g; s/$REPLACELINE1\n$REPLACELINE2/$REPLACEWITHLINE2/ p }" /tmp/mainscan > /tmp/edscan


or this
Code:
sed -n '
/^[Encryption[]]/ {
p
n
/^[[Bit]]/ {
i \
ESSID:""
p}
}
/^[^[Bit]]/ p' /tmp/mainscan

might work with some adjustments!
Back to top
View user's profile Send private message MSN Messenger 
stemsee

Joined: 27 Jun 2013
Posts: 2260
Location: In The Way

PostPosted: Mon 09 Apr 2018, 17:31    Post subject:  

This line finds it
Code:
awk '{if ($0 ~ /Encryption key:/) {triggered=1;}if (triggered) {print; if ($0 ~ /Bit Rates:1/) { exit;}}}' /tmp/mainscan


Now just to figure out how to replace found pattern with the inserted line!
Back to top
View user's profile Send private message MSN Messenger 
musher0


Joined: 04 Jan 2009
Posts: 12569
Location: Gatineau (Qc), Canada

PostPosted: Mon 09 Apr 2018, 17:46    Post subject:  

Feedback, FWIW, IMO:

"Bit rates" lines are identical, so you do not need to scan for them;
on or off does not matter so you only need to check for the expression
< Encryption key >, as in
Code:
[ "${line%\:*}" = "encryption key" ] && echo -e "$line\n"ESSID:\"\"\n" >> receiving.lst

I am not sorry that I do not know sed, since I get by easily in these problems
with awk and bash's string detection and substitution functions. (I read some-
where that these in-built bash functions are faster than either awk or sed.)

BFN.

_________________
musher0
~~~~~~~~~~
Siempre será canción nueva... (V. Jara, Manifiesto)
Back to top
View user's profile Send private message 
step

Joined: 04 May 2012
Posts: 1066

PostPosted: Mon 09 Apr 2018, 17:49    Post subject:  

Untested awk script:
Code:

/^Encryption key:off/
# if the current line matched the above pattern
{
  got1 = NR # save the line number
  print # print the line
  next # skip the rest of this script
}

/^Bit Rates:1 Mb\/s; 2 Mb\/s; 5.5 Mb\/s; 11 Mb\/s; 6 Mb\/s/
# if the current line matched the above pattern
{
  if the line number is 1+the Encryption line number
  if (NR == got1+1) {
    print "ESSID:\"\"" # insert/print the thing
  }
  print # print the line
  next  # skip the rest of this script
}

{
  print  # if we get here just print the current line
}
# over and over again till the end of the file.

Put this script in a file, say script.awk and invoke it with
awk -f script.awk input-file.

Suggestion: try to simplify the patterns. Ask yourself what's the shortest pattern that uniquely identifies the insertion point?

edited to improve formatting

_________________
Fatdog64-721|+Packages|Kodi|Findnrun|+forum|gtkmenuplus

Last edited by step on Mon 09 Apr 2018, 18:05; edited 1 time in total
Back to top
View user's profile Send private message 
musher0


Joined: 04 Jan 2009
Posts: 12569
Location: Gatineau (Qc), Canada

PostPosted: Mon 09 Apr 2018, 17:54    Post subject:  

stemsee wrote:
This line finds it
Code:
awk '{if ($0 ~ /Encryption key:/) {triggered=1;}if (triggered) {print; if ($0 ~ /Bit Rates:1/) { exit;}}}' /tmp/mainscan

Now just to figure out how to replace found pattern with the inserted line!
May I suggest a more detailed approach? It will allow you to (re)print the parts
of the expression and then add a line feed plus the string that you need.
Code:
awk '{if ($1 ~ /Encryption/ && $2 ~ /key:/) {triggered=1;}if (triggered) {print $1,$2"\nESSID:\"\"\n"; if ($0 ~ /Bit Rates:1/) { exit;}}}' /tmp/mainscan

I'll keep my fingers crossed... Smile

_________________
musher0
~~~~~~~~~~
Siempre será canción nueva... (V. Jara, Manifiesto)
Back to top
View user's profile Send private message 
stemsee

Joined: 27 Jun 2013
Posts: 2260
Location: In The Way

PostPosted: Tue 10 Apr 2018, 02:36    Post subject:  

Thanks @step @musher0

I will play with these.

musher0 the problem is that there are two Bit Rates: lines per Cell found;
also every Cell has Encryption key: line too; between these two lines there is
usually ESSID:"*" line; sometimes that line does not exist and so the script needs to add a dummy "".
Therefore only finding Encryption line and printing ESSID after it might add
a second ESSID line; same for adding ESSID line before Bit Rates line - as
there are two such lines, which means three ESSID: lines will exist for each
Cell (I already achieved that before posting).

Your last solution might work but not the previous ones. I will test it.

step's solution seems to include all these conditional requirements; including multiple occurrences.

thanks again
stemsee
Back to top
View user's profile Send private message MSN Messenger 
stemsee

Joined: 27 Jun 2013
Posts: 2260
Location: In The Way

PostPosted: Tue 10 Apr 2018, 03:02    Post subject:  

Neither work!

muserh0's script has an unexpected '(' error ... tried to parse it o no avail.

I redirected step's script to a new file in tmp but no print occurred.
Code:
/^Encryption key:/
# if the current line matched the above pattern
{
  got1 = NR # save the line number
  print # print the line
  next # skip the rest of this script
}

/^Bit Rates:1/
# if the current line matched the above pattern
{
 # if the line number is 1+the Encryption line number
  if (NR == got1+1) {
    print "ESSID:\"\"" # insert/print the thing
  }
  print # print the line
  next  # skip the rest of this script
}

{
  print  # if we get here just print the current line
}
# over and over again till the end of the file.

awk -f /usr/sbin/scanfix.awk < /tmp/mainscan > /tmp/new
Back to top
View user's profile Send private message MSN Messenger 
MochiMoppel


Joined: 26 Jan 2011
Posts: 1579
Location: Japan

PostPosted: Tue 10 Apr 2018, 05:51    Post subject:  

stemsee, we had this problem before: Please remove the insanely long strings (IE: Unknown:xxx) from your code. Doubles the width of the forum page (I hate horizontal scrolling).

As for the replacement, the following code works for me. Would be much easier if you would clean all leading whitespace, but anyway, the code retains the whitespace.

Code:
#!/bin/bash
SCANRESULT='         Cell 01 - Address: 2E:FE:4A:5A:48:1B
                    Channel:13
                    Frequency:2.472 GHz (Channel 13)
                    Quality=49/70  Signal level=-61 dBm 
                    Encryption key:on
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s
                              9 Mb/s; 12 Mb/s; 18 Mb/s
                    Bit Rates:24 Mb/s; 36 Mb/s; 48 Mb/s; 54 Mb/s
                    Mode:Master
          Cell 02 - Address: 90:21:06:24:35:71
                    Channel:1
                    Frequency:2.412 GHz (Channel 1)
                    Quality=31/70  Signal level=-79 dBm 
                    Encryption key:on
                    ESSID:"NOWTV65791"
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 18 Mb/s
                              24 Mb/s; 36 Mb/s; 54 Mb/s
                    Bit Rates:6 Mb/s; 9 Mb/s; 12 Mb/s; 48 Mb/s
                    Mode:Master
          Cell 03 - Address: 70:50:AF:A9:23:92
                    Channel:6
                    Frequency:2.437 GHz (Channel 6)
                    Quality=20/70  Signal level=-90 dBm 
                    Encryption key:off
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 18 Mb/s
                              24 Mb/s; 36 Mb/s; 54 Mb/s
                    Bit Rates:6 Mb/s; 9 Mb/s; 12 Mb/s; 48 Mb/s
                    Mode:Master'

echo "$SCANRESULT" | tr '\n' '\x01' | sed -r '
s/(Encryption key:o[nf]*\x01)([[:blank:]]*)(Bit)/\1\2ESSID=""\x01\2\3/g
s/\x01/\n/g'


Result:
Code:
         Cell 01 - Address: 2E:FE:4A:5A:48:1B
                    Channel:13
                    Frequency:2.472 GHz (Channel 13)
                    Quality=49/70  Signal level=-61 dBm 
                    Encryption key:on
                    ESSID=""
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s
                              9 Mb/s; 12 Mb/s; 18 Mb/s
                    Bit Rates:24 Mb/s; 36 Mb/s; 48 Mb/s; 54 Mb/s
                    Mode:Master
          Cell 02 - Address: 90:21:06:24:35:71
                    Channel:1
                    Frequency:2.412 GHz (Channel 1)
                    Quality=31/70  Signal level=-79 dBm 
                    Encryption key:on
                    ESSID:"NOWTV65791"
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 18 Mb/s
                              24 Mb/s; 36 Mb/s; 54 Mb/s
                    Bit Rates:6 Mb/s; 9 Mb/s; 12 Mb/s; 48 Mb/s
                    Mode:Master
          Cell 03 - Address: 70:50:AF:A9:23:92
                    Channel:6
                    Frequency:2.437 GHz (Channel 6)
                    Quality=20/70  Signal level=-90 dBm 
                    Encryption key:off
                    ESSID=""
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 18 Mb/s
                              24 Mb/s; 36 Mb/s; 54 Mb/s
                    Bit Rates:6 Mb/s; 9 Mb/s; 12 Mb/s; 48 Mb/s
                    Mode:Master
Back to top
View user's profile Send private message 
stemsee

Joined: 27 Jun 2013
Posts: 2260
Location: In The Way

PostPosted: Tue 10 Apr 2018, 07:57    Post subject:  

Thanks @ MochiMoppel

Code:
iwlist scan wlan0 | grep -v -e 'IE: Unknown' > /tmp/mainscan


Code:
SCANRESULT=$(cat /tmp/mainscan)
echo "$SCANRESULT" | tr '\n' '\#01' | sed -r '
s/(Encryption key:o[nf]*\#01)([[:blank:]]*)(Bit)/\1\2ESSID=""\#01\2\3/g
s/\#01/\n/g' | tr '\#' '\n' > /tmp/fixscan

then I had to translate back so I swapped 'x' for '#'

but it didn't work for me on gnu sed 4.2.2 x64 on fatdog neither in term or redirected to new file.

I must be doing something wrong, perhaps copying and pasting from the forum.

Last edited by stemsee on Tue 10 Apr 2018, 12:07; edited 1 time in total
Back to top
View user's profile Send private message MSN Messenger 
stemsee

Joined: 27 Jun 2013
Posts: 2260
Location: In The Way

PostPosted: Tue 10 Apr 2018, 08:22    Post subject:  

So I am finding a solution simmilar to musher0's

Code:
awk '$0 ~ "Encryption" {print NR} $0 ~ "Bit Rates:1" {print NR}' /tmp/mainscan | while read line1; do read line2; echo "$(($line2 - 1)) $line1"; done
7 6
23 22
38 38
54 53
66 65

so this gives the line numbers and does a subtraction to show that pattern one and two are one line apart instead of two, which means a line can be inserted after $line1.
Back to top
View user's profile Send private message MSN Messenger 
stemsee

Joined: 27 Jun 2013
Posts: 2260
Location: In The Way

PostPosted: Tue 10 Apr 2018, 09:17    Post subject:  

This is working!
Code:
awk '$0 ~ "Encryption" {print NR} $0 ~ "Bit Rates:1" {print NR}' /tmp/mainscan | while read line1; do read line2; echo "$(($line2 - 1)) $line1"; done > /tmp/this
while read line
do
pato=$(echo $line | cut -f1 -d' ')
patt=$(echo $line | cut -f2 -d' ')
if [[ "$pato" = "$patt" ]]; then
   inspat=$((patt + 1))
   sed -i "${inspat}iESSID:##" /tmp/mainscan
   sed -i 's/##/""/g' /tmp/mainscan
fi
done < /tmp/this
Back to top
View user's profile Send private message MSN Messenger 
MochiMoppel


Joined: 26 Jan 2011
Posts: 1579
Location: Japan

PostPosted: Tue 10 Apr 2018, 10:31    Post subject:  

stemsee wrote:
I must be doing something wrong
Sure, you changed \x01 to \#01. Why? Makes no sense. Please note that '\x01' is interpreted as a single character (ASCII 01)
You also added a final tr command which not only tries to replace the wrong characters but is also unnecessary. Changing back the placeholder for the newline character is already taken care of by the sed command.

P.S.: I didn't mean that you grep for 'IE: Unknown'. I only meant that you make the demo output in your first post shorter to fit the forum page. Your actual output may be as long and wide as it needs to be, however to demonstrate your problem a shortened version of the output is sufficient and helps to grasp the issue.
Back to top
View user's profile Send private message 
stemsee

Joined: 27 Jun 2013
Posts: 2260
Location: In The Way

PostPosted: Tue 10 Apr 2018, 12:19    Post subject:  

I tried it out before editing. But in term and in file nothing translated back to '\n' so was unparsable.

I will try it in script with 'bash' instead of sh. My system often disagrees!!

So I will test it out on another operating system, namely JustLightouse and artful/bionic bever and DebianDog xenial.

cheers
stemsee
Back to top
View user's profile Send private message MSN Messenger 
Display posts from previous:   Sort by:   
Page 1 of 2 [21 Posts]   Goto page: 1, 2 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Off-Topic Area » Programming
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.0888s ][ Queries: 13 (0.0050s) ][ GZIP on ]