Gethosts script

A home for all kinds of Puppy related projects
Post Reply
Message
Author
jockjunior
Posts: 150
Joined: Tue 29 Jan 2008, 21:11
Location: Lancashire,U.K.
Contact:

Gethosts script

#1 Post by jockjunior »

Hi all, downloaded Debiandog and am very impressed with it Saintless, some good work done there.
this script I found on the internet but have amended it to work in DebianDog. Sounds impressive but all it took was to alter the directories, and remove the sudo commands to suit DebianDog. It should work with any Puppy as well . Place the script in /bin and make it executable.

#!/bin/bash

# If this is our first run, save a copy of the system's original hosts file and set to read-only for safety
if [ ! -f /root/hosts-system ]
then
echo "Saving copy of system's original hosts file..."
cp /etc/hosts /root/hosts-system
chmod 444 /root/hosts-system
fi

# Perform work in temporary files
temphosts1=$(mktemp)
temphosts2=$(mktemp)

# Obtain various hosts files and merge into one
echo "Downloading ad-blocking hosts files..."
wget -nv -O - http://winhelp2002.mvps.org/hosts.txt >> $temphosts1
wget -nv -O - http://hosts-file.net/ad_servers.asp >> $temphosts1
wget -nv -O - http://someonewhocares.org/hosts/hosts >> $temphosts1
wget -nv -O - "http://pgl.yoyo.org/adservers/serverlis ... =plaintext" >> $temphosts1

# Do some work on the file:
# 1. Remove MS-DOS carriage returns
# 2. Delete all lines that don't begin with 127.0.0.1
# 3. Delete any lines containing the word localhost because we'll obtain that from the original hosts file
# 4. Replace 127.0.0.1 with 0.0.0.0 because then we don't have to wait for the resolver to fail
# 5. Scrunch extraneous spaces separating address from name into a single tab
# 6. Delete any comments on lines
# 7. Clean up leftover trailing blanks
# Pass all this through sort with the unique flag to remove duplicates and save the result
echo "Parsing, cleaning, de-duplicating, sorting..."
sed -e 's/\r//' -e '/^127.0.0.1/!d' -e '/localhost/d' -e 's/127.0.0.1/0.0.0.0/' -e 's/ \+/\t/' -e 's/#.*$//' -e 's/[ \t]*$//' < $temphosts1 | sort -u > $temphosts2

# Combine system hosts with adblocks
echo Merging with original system hosts...
echo -e "\n# Ad blocking hosts generated "$(date) | cat /root/hosts-system - $temphosts2 > /root/hosts-block

# Clean up temp files and remind user to copy new file
echo "Cleaning up..."
rm $temphosts1 $temphosts2
echo "Done."
echo
echo "Copy ad-blocking hosts file with this command:"
echo " cp /root/hosts-block /etc/hosts"
echo
echo "You can always restore your original hosts file with this command:"
echo " cp /root/hosts-system /etc/hosts"
echo "so don't delete that file! (It's saved read-only for your protection.)"
echo

hope it's of interest to someone

Phil (old git silver surfer)

edit it puts the files hosts-block and hosts-system in folder /root

User avatar
LazY Puppy
Posts: 1934
Joined: Fri 21 Nov 2014, 18:14
Location: Germany

#2 Post by LazY Puppy »

Shouldn't this more related being in the Programming section?
RSH

"you only wanted to work your Puppies in German", "you are a separatist in that you want Germany to secede from Europe" (musher0) :lol:

No, but I gave my old drum kit away for free to a music store collecting instruments for refugees! :wink:

User avatar
drunkjedi
Posts: 882
Joined: Mon 25 May 2015, 02:50

#3 Post by drunkjedi »

I believe the built in gui app, pup-advert-blocker use similar script.
I thought Debian dog had it too.

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

Re: Gethosts script

#4 Post by saintless »

Hi jockjunior.

The script attached as debdog-advert-blocker in zip archive with modifications for multiuser (it will save in $HOME/ instead in /root and message how to run the script as user).
I can make deb package in a few days with menu entries for debdog repository but I prefer to have the source mentioned in the script. Can you give link to it?

BTW pup-advert-blocker seems to work from quick testing in DebianDog without modifications after installing dos2unix package.

Edit: The archive removed. Here is download link for the deb package:
http://kazzascorner.com.au/saintless/De ... 1_i386.deb

Toni
Last edited by saintless on Fri 15 Apr 2016, 08:22, edited 2 times in total.

mcewanw
Posts: 3169
Joined: Thu 16 Aug 2007, 10:48
Contact:

#5 Post by mcewanw »

Maybe from here, Toni:

https://www.kubuntuforums.net/showthrea ... hosts-file

EDIT: Someone's mods to the script, which he used on a router, allowed the creation of a whitelist of sites not wanting blocked (for some people still wanting google-analytics to work I think):

https://www.kubuntuforums.net/showthrea ... nux-router

William
github mcewanw

jockjunior
Posts: 150
Joined: Tue 29 Jan 2008, 21:11
Location: Lancashire,U.K.
Contact:

#6 Post by jockjunior »

Hi Toni, here's where i got the original script from

https://www.kubuntuforums.net/showthrea ... hosts-file

sorry for delay in replying been away from computer

Phil

User avatar
saintless
Posts: 3862
Joined: Sat 11 Jun 2011, 13:43
Location: Bulgaria

#7 Post by saintless »

jockjunior wrote:Hi Toni, here's where i got the original script from

https://www.kubuntuforums.net/showthrea ... hosts-file
Thanks Phil, I will make deb package in the next few days.
I will post the information in DD thread when I'm ready.
mcewanw wrote:EDIT: Someone's mods to the script, which he used on a router, allowed the creation of a whitelist of sites not wanting blocked (for some people still wanting google-analytics to work I think):

https://www.kubuntuforums.net/showthrea ... nux-router
Thanks William. I think to include only the attached zip archive script in the package. It is simple and doesn't change /etc/hosts automatically + option to restore easy the original file later. No problem for me if someone likes to improve it or add extra scripts in the package later.

Edit: Download link for debdog-advert-blocker_1.0.0-1_i386.deb
The package contains two scripts in /usr/local/bin with menu entry Settings -> DebDog-advert-blocker
One of the scripts is unchanged original version gethosts.
The second script debdog-advert-blocker executes gethosts opening terminal window. Starts gethosts with "sudo -E" for user account or without sudo for root.

Toni
Attachments
debdog-advert-blocker.png
(106.99 KiB) Downloaded 276 times

LesStrater
Posts: 2
Joined: Wed 15 Jun 2016, 22:47

Re: Gethosts script

#8 Post by LesStrater »

This script is exactly what I've been looking for, with minor modification. I've used the MVPS hosts file for years to block ad content, and this script will automate the process of downloading and installing updated versions of it every month. The MVPS hosts file (IMHO) is the best one out there.

I did find one major bug in the script: The script states that it will "Delete all lines that don't begin with 127.0.0.1 -- and it correctly does that. The problem is that all the server addresses in the MVPS hosts file begin with with local host "0.0.0.0", not "127.0.0.1". This means that the script downloads the MVPS list and then promptly deletes all the entries in it.
I tried for hours to modify that convoluted SED command to also include "0.0.0.0" from its exclusion parameter, but I'm not a strong enough programmer, and none of the examples I found online worked properly. In the end, I wound up using two SED commands, one to delete each local machine address, then I appended the two lists into one.

Additionally, I deleted the downloading of the HUGE hosts file from "hosts-file.net". Whoever maintains this list has some overzealous political/moral motivation behind their efforts. I found several video-streaming servers which I regularly use being blocked by this list. I'm not talking porn, but regular movie and TV show sites like Primewire. (I did also notice MANY porn servers on the list--for those of you who care.) I'm interested in ad-blocking, not content-blocking...

Thanks to jockjunior (Phil) for making this script available!

Post Reply