Normal Linux commands to Locate your system files, INSTANTLY

[slavvo67]

Hi Musher0:

The directory issue is a tough call. It might be best to have the directory file in a location that doesn't get lost on shutdown by live cd users. Even so, one could argue that the mounts might not always be the same at next boot; an apparent problem with saving for another day.

Personally, it doesn't matter to me where it goes but I prefer to know where it places itself for future reference, deletion, etc.

Attached Drive_Index_Creator-4. It allows for a rm (remove) option once a sub-search is completed.

Best,

Slavvo67

[gcmartin]

... What I'd like to know is what most users think ...

As is the case with SUSE/Fedora/etc, this should be a system folder versus a user folder. This would reduce any developer/support/user who might need help. Stored in the same folder on every PUP distro.

If you have Lighthouse64 running, take a look at its.

I hope this is what you were looking for.

the mounts might not always be the same at next...

Good point. This should have an initial cron job and maybe an option in FirstRUN to raise user awareness so that they can take some reasonable step(s) to control their personal systems appropriately. I think we can construct a simple sentence or a mouse-over the selectable option, if placed there. FirstRUN raises awareness, no matter how experienced any user is.

Just some ideas...

[musher0]

(...)

Code: Select all

slocate slocate.db 
/var/lib/slocate/slocate.db

instant results.

Ha-ha, so there it is!
(This ha-ha doesn't work written, it needs the proper tone of voice!)
Thanks, Puppus.

[musher0]

@Slavvo67:

I took a look at your code (v. 3., I haven't had a chance to look at your v. 4 yet.):
nice job with the if-elseif's ! You taught me something!

I also saw that you use find rather than slocate or ls to build your db. I tested
and I now know for a fact that find is a few milliseconds faster than < ls -R > for
indexing folder ~/my-applications on my machine.

That said, I have a problem, not with your code, not at all, by with the single
entry approach, whether the developer uses find or < ls -R >.

When the user consults the db, through grep or whatever, how is he or she
to distinguish between file "blah" in /usr/share/doc and file "blah" in /usr/bin,
if the listing contains only one column? We must remember that there are
thousands and thousands of files on a Linux system. So there needs to be a way
to inform the user of the context, no?

Even with the grep context parameters -A x and -B y, sometimes you do not see
in which directory a file is located. And of course it would be nonsense to ask
grep to find file "blah" with a context of say 50 lines before and 50 lines after.
Oh, grep no doubt can do it, I mean that it would be nonsense to impose 100
lines of context to the user.

One nice solution would be to fire up the files db in the less reader, but less is
its own kettle of fish! Plus bye-bye speed of consultation...

My 2-3 ¢ worth!

BFN.

musher0

[smokey01]

What I really like about slocate is it's speed. It's actually quite fast. When it does an update it's even faster as it seems to do a differential update.

I have heaps of drives and partitions and I put files all over the place and always have trouble finding them. This is why I need one database to contain all the files or it becomes difficult to know where to start searching with numerous databases.

It's easy enough to mount all drives and index them at boot/reboot.

The output needs to show drive/partition and the full pathname so the file can be found.

The next thing required is a simple gui for people with cliphobia.

[Uten]

Smoky01 et.all,

Considering indexing all disks and create one "huge" db file or collecting several files accessible on your primary disk. A humble suggestion is to prepend locate.db with the disk/partition's LABEL or UUID. you get the LABEL and UUID with the command blkid. Collect the files in /var/<somewhere> and search through them with grep.

[musher0]

What I really like about slocate is it's speed. It's actually quite fast. When it does an update it's even faster as it seems to do a differential update.

I have heaps of drives and partitions and I put files all over the place and always have trouble finding them. This is why I need one database to contain all the files or it becomes difficult to know where to start searching with numerous databases.

It's easy enough to mount all drives and index them at boot/reboot.

The output needs to show drive/partition and the full pathname so the file can be found.

The next thing required is a simple gui for people with cliphobia.

Hi, smokey01.

Did you just say that you got slocate to index outside drives?
I must be a cluts, 'cause I can't seem to do it? What's your trick?

Thanks in advance.

musher0

[gcmartin]

Hello @Uten
I think what @Smokey01 is sharing, "is that maybe largely unnecessary". I quite agree.

But, this does NOT change the fact that some will want to extend the ability to isolate some info found in the database into some individual categories.

@Smokey01 and others who may have an ear to those who exist in the WOOFCE/WOOFQ/etc arena where PUPs get built, what is necessary to get this message to them, especially now that PUPs are being built for decade-old PCs which have large GB and now TB HDDs/USBs/SSDs/microSDs/etc. in their PCs.

Let hope the message is received warmly by the PUP builders.

With this as a terminal command in all new PUPs, a simple "non-terminal" search screen can be easily crafted. Or further, pFind/PSearch might return results faster with a simple mod. Or, ...

Now how do we get this knowledge to the builders?
(I think you must be a registered Puppy GIT user in order to present "anything" to the WOOFCE/WOOFQ builders. If not, there is no forum manner and we can only hope that one of the registered users will present the need to them.)

edit: Corrected the Pup app's name from pSearch to pFind

[smokey01]

Did you just say that you got slocate to index outside drives?

Not yet but it's what I need.

[musher0]

Smoky01 et.all,

Considering indexing all disks and create one "huge" db file or collecting several files accessible on your primary disk. A humble suggestion is to prepend locate.db with the disk/partition's LABEL or UUID. you get the LABEL and UUID with the command blkid. Collect the files in /var/<somewhere> and search through them with grep.

Hi, uten.

You put your finger on something when you say "one 'huge' db file". Slavvo67's utility
does that. Like many of you I have many files on my drives, and the size of the db
created was 41 Mb's.

That's a no-no for Puppy philosophy in general and certainly for Puppyists running
their Puppy from CD. (The CD will just choke.)

Using one huge file excludes /var/somewhere. Rather maybe store the one db on
/mnt/sda1 if we go the "one large db file" route.

As for me I'd prefer one db per drive with symlinks in a central place. With symlinks
it could indeed be /var/somwhere, but not with a humongous file. IMO, the symlinks
approach can be just as efficient if we can give grep or another searching tool the
proper searching parms.

(Using the block id's for the separate drive file may be a good idea indeed.)

My 2 ¢. BFN.

musher0

[gcmartin]

Hello @Musher0, and I agree with everything you said, assuming you are using an older PC with not very much RAM and certainly no SWAP.

But, I am still inclined to understand that as, in your case, 41MB file is more than your PC can bear, but, for all of my PCs circa 2000+, I would not have any problem with the tradeoff for instantaneous searches versus a small db managed in system's RAM (for those PUPPY members who run Frugal/CD/DVD. In full PUP implementations, this is a non-issue).

Like all other things in Linux's filesystem, when SWAP is employed, anything not utilized recently will be candidates for migration to the SWAP device, thus, again, the impact of the db is minimized.

I am not suggesting that this terminal command be backward retrofitted in older PUPs, instead, I am wondering if consideration is prudent to add this to PUP commandset for new distros so that users/planners can determine how they see the implementation in their running systems.

I do agree with you and others, that, if possible, it would be nice to have a feature which would allow categories of data to exist (whether that be mounted devices or videos or audios or etc). But I feel we need our development community to acknowledge the need for this command in PUPPY's arsenal.

We have far too many files for those of us who have used PUPPY LInux over the last decade to continue to experience the time lag when trying to grab something for development/need. This practically involves ALL OF US.

Fingers crossed ...

[musher0]

Hi gc.

I never was a bureaucrat, never will be. I trip on my own toes if you give me a form to
fill. Same kind of allergy as Smokey01's towards cli's, except for forms.

This thread / project needs a good secretary / salesman / "pimp" which the devs can
team up with / rely on.

BFN.

musher0

[smokey01]

I have found a solution that fits my requirements. It's called mlocate, not slocate or locate.

If your drives are mounted, it will index them to a single mlocate.db file. An odd thing though, if your drive is not mounted it doesn't seem to display below / of that drive.

It is suppose to be backward compatible with both locate and slocate. The binary is actually called locate just to make things a little confusing. Do a mlocate -V and it will display the version.

The default database pathname is:
/usr/var/mlocate/mlocate.db

This can be changed with a mlocate.conf file I believe although I have not tried it.

musher0, I don't mind the CLI but many don't.

See what you think of this package.

[Puppus Dogfellow]

[...]

Did you just say that you got slocate to index outside drives?
I must be a cluts, 'cause I can't seem to do it? What's your trick?

Thanks in advance.

musher0

http://linux.about.com/library/cmd/blcmdl1_slocate.htm
Linux / Unix Command: slocate
Command Library
NAME
slocate - Security Enhanced version of the GNU Locate
SYNOPSIS

slocate [-qi] [-d <path>] [--database=<path>] <search string>

slocate [-i] [-r <regexp>] [--regexp=<regexp>]

slocate [-qv] [-o <file>] [--output=<file>]

slocate [-e <dir1,dir2,...>] [-f <fstype1,...>] <[-l <level>] [-c] <[-U <path>] [-u]>

slocate [-Vh] [--version] [--help]


DESCRIPTION

Secure Locate provides a secure way to index and quickly search for files on your system. It uses incremental encoding just like GNU locate to compress its database to make searching faster, but it will also store file permissions and ownership so that users will not see files they do not have access to.


This manual page documents the GNU version of slocate. slocate Enables system users to search entire filesystems without displaying unauthorized files.
OPTIONS

-u
Create slocate database starting at path /.
-U <dir>
Create slocate database starting at path <dir>.
-e <dir1,dir2,...>
Exclude directories from the slocate database.
-f <fstype1,...>
Exclude files on specific file systems from the slocate database.
-c
Parse '/etc/updatedb.conf' when updating the slocate database.
-l <level>
Security level. 0 turns security checks off. This will make searchs faster. 1 turns security checks on. This is the default.
-i
Does a case insensitive search.
-q
Quiet mode. Error messages are suppressed.
-n <num>
Limit the amount of results shown to <num>.
-r <regexp>
--regexp=<regexp> Search the database using a basic POSIX regular expression.
-o <file>
--output=<file> Specifies the database to create.
-d <path>
--database=<path> Specifies the path of databases to search in.
-h
--help Display this help.
-v
--verbose Verbose mode. Display files when creating database.
-V
--version Display version.

any of this help, musher? it seems you can tell it to create databases for anything and place them anywhere, if i'm not misreading the code, entirely likely though that may be.

i'm not clear on whether it automatically picks up on the locations of the databases it's made/whether you specify them just to save time.

do your scripts rebuild the database each time or do they get updated with a difference file of sorts?

thanks in advance.

[musher0]

Hi gc.

Aswers interspersed in the quote of your post.

Hello @Musher0, and I agree with everything you said, assuming
you are using an older PC with not very much RAM and certainly no SWAP.

But, I am still inclined to understand that as, in your case, 41MB file is more than
your PC can bear, but, for all of my PCs circa 2000+, I would not have any problem
with the tradeoff for instantaneous searches versus a small db managed in
system's RAM (for those PUPPY members who run Frugal/CD/DVD. In full PUP
implementations, this is a non-issue).

Of course in FULL intall, it's a non issue, but we need to find a location (drive/
folder) that fits all situations (Full/Frugal/CD/DVD). An intelligent choice will save
us from writing more code lines.

Like all other things in Linux's filesystem, when SWAP is employed, anything not
utilized recently will be candidates for migration to the SWAP device, thus, again,
the impact of the db is minimized.

Having a SWAP file prevents the OS from crashing, but if it gets used even slightly,
a car jam at rush hour downtown may be sheer fun compared to it. When the
system uses the SWAP file, It really slows you down.

I am not suggesting that this terminal command be backward retrofitted in
older PUPs, instead, I am wondering if consideration is prudent to add this to PUP
commandset for new distros so that users/planners can determine how they see the
implementation in their running systems.

See my post above: this thread / project needs a "pimp" / secretary / salesman to
handle the submitting of forms to the github.

I do agree with you and others, that, if possible, it would be nice to have a feature
which would allow categories of data to exist (whether that be mounted devices or
videos or audios or etc)...

Specialized "locate" applications already exist for "mounted devices or videos or
audios or etc", e.g. DataCrow. This is the only filename that comes to my mind at
present (it's probably the best, too), but search the Internet a bit and you'll find
that there are countless others for music CD's, film DVD's, USB contents, etc., etc.

...But I feel we need our development community to acknowledge the need for this
command in PUPPY's arsenal.

We have far too many files for those of us who have used PUPPY LInux over the last
decade to continue to experience the time lag when trying to grab something for
development/need. This practically involves ALL OF US.

Fingers crossed ...

Of course, there's a need. We just have to fill it properly the first time. Don't you
hate it when you have to do something over?

Of course, we need the community to acknowledge the need -- if they don't they're
all schizo's! And that's when -- Act 2, Scene 1 -- the "pimp" / secretary /
salesman (or saleswoman) enters the scene!

BFN.

musher0

[musher0]

Ha-ha again, Puppus.

I think that I've been using a different animal :

Secure Locate 3.1 - Released March 7, 2006
Copyright (c) 2005 Kevin Lindsay

Search: slocate [-qi] [-d <path>] [--database=<path1:path2:...>]
<search string>
slocate [-r <regexp>] [--regexp=<regexp>]
Update database: slocate [-qv] [-o <file>] [--output=<file>]
slocate [-e <dir1,dir2,...>] [-f <fs_type1,...> ] [-l <level>]
[-c <file>] <[-U <path>] [-u]>
General: slocate [-Vh] [--version] [--help]

Options:
-u - Create slocate database starting at path /.
-U <dir> - Create slocate database starting at path <dir>.
-c <file> - Parse original GNU Locate's configuration file
when using the -u or -U options. If 'updatedb' is
symbolically linked to the 'slocate' binary, the
original configuration file '/etc/updatedb.conf' will
automatically be used.
-e <dir1,dir2,...> - Exclude directories from the slocate database when
using the -u or -U options.
-f <fs_type1,...> - Exclude file system types from the slocate database
when using the -u or -U options. (ie. NFS, etc).
-l <level> - Security level.
0 turns security checks off. This will make
searchs faster.
1 turns security checks on. This is the default.
-q - Quiet mode. Error messages are suppressed.
-n <num> - Limit the amount of results shown to <num>.
-i - Does a case insensitive search.
-r <regexp>
--regexp=<regexp> - Search the database using a basic POSIX regular
expression.
-o <file>
--output=<file> - Specifies the database to create.
-d <path>
--database=<path> - Specfies the path of databases to search in.
-h
--help - Display this help.
-v
--verbose - Verbose mode. Display files when creating database.
-V
--version - Display version.

Author: Kevin Lindsay
Bugs: slocate@trakker.ca
HTTP: http://slocate.trakker.ca/

Many thanks. (EDIT: sentence removed.)
Now to hunt down your slocate...

BFN.

musher0

[musher0]

@Puppus, and also Smokey01.

It says "incremental" on the GNU slocate man page, DESCRIPTION section, 2nd line,
1st word. That sure explains the speed of updates.

BFN.

musher0

[musher0]

Tempting...

http://rlocate.sourceforge.net/
https://fedorahosted.org/mlocate/

[musher0]

Moved to: http://www.murga-linux.com/puppy/viewto ... ost#830109

[Puppus Dogfellow]

Ok, folks.

Drums rolling, curtains up... I give you: PuppyTerrier-01! For you to test!
I think that, in this *pet, I've solved all the "spiney" points we've uncovered so far.

* First, my apologies to Smokeyp01, who doesn't get a GUI...

__ However we have ANSI colors, and IMO, there's basically no difference between
__ typing in a GUI and typing in a CLI window, since typing is typing, eh?

__ The allergenic CLI aspect (for some people) is, I believe, largely mitigated by
__ the use of *.desktop files. Two scripts will appear in your menu, one for
__ constructing/updating the db's and one for searching them.

* the external drive db's stay on the external drives, so there's minimal hit on the
__ internal Puppy size. Only the index of the Puppy internal files is stored internally.
__ This feature makes slocate available for all types of Puppy configurations, either
__ full, frugal or CD/DVD. I paid particular attention to this feature.

* the search script searches everything each time, the Puppy OS and the mounted
__ external drives.

* updating takes 2-3 seconds max on my rig (I have +/- 165 Mg's worth of files on
__ 8 partitions.)

* the automatic launch through Startup has been "niced", using Uten's hint and
__ K. Lindsay's (the author of slocate) own one-liner for "croning" slocate.

* Speaking of which, the original slocate config. and script in /etc have been
__ removed, we don't need them with Médor's script.

* As a bonus, Médor's RAM-refreshing line is included in the automatic update.
__ Just comment that line with a "#" if you don't want it or need it. But it's there,
__ I for one find it quite useful. Also...
__ More time intervals have been included in that script. You may want to test what
__ time interval is more convenient for you. (There's a how-to in the script.)

* I thank you all of course, for the fruitful ideas and exchanges you provided in this
__ thread. Thanks are obviously due to the author of slocate, Kevin Lindsay, and to
__ gcmartin, initiator of this thread; also to Slavvo67, Uten and Médor for their
__ respective hints and contributions. Thanks also to Puppus Dogfellow who gave me
__ the little push I needed when I was stumped.

So, please test the attached to your heart's content. Constructive feedback will be
most welcome and will make this little utility even better.

Enjoy!

musher0

cool, musher. works well and is very fast. is there a way to read the .db files as text or html, or a way to convert the databases to something a browser or text editor can read? could you add (something like) output to file.txt as a choice for each of the databases created?

thanks in advance.