Google-Fu

[Strolen]

Since it has been alluded to many-a-time and the power of your Google-Fu can greatly enhance your online prowness, I figured it was high time that there was a thread to help expand this skill.

Solid Tips To Live By:
(if you are brave, try and browse through the rest of the site)
Fravia's 16 quick searching tips
I think by simply following a couple of these tips you will instantly increase your Google-Fu skills.

Two links that summarize some of googles powerful operators that come in handy for those specific searches.

Quick Reference: Google Advanced Operators (Cheat Sheet)
Using Search Operators (Advanced Operators)

That is actually enough for one post.

[MoonHunter]

The framed paper doors slide open. The students file in. They take their positions, focusing on the Si-fu.

"When one studies Kung-Fu, one develops ones tool- ones body- to handle the rigors of the manuvers. The drills not only pattern the motions, but develop flexibility and stamina... as well as some secondary mental techniques (soft focus and projected balance).  

For Google-Fu you must also develop your tool... this case your mind.

Athleticism for Google-Fu is actually speed reading. The faster you can scan a page for what you are looking for, the faster you can get that information."

Of course there are links....

http://english.glendale.cc.ca.us/methods.html

http://en.wikibooks.org/wiki/Speed_reading

http://www.memletics.com/speed-reading-course/default.asp?ref=ga&data=usd+speed+reading+focused

http://il.essortment.com/speedreadingte_rjpv.htm

There are dozens more, but you will have to find them yourselves.

http://www.ucc.vt.edu/stdysk/suggest.html

[MoonHunter]

The framed paper doors slide open. The students were hard at work with their eye drills. The Sifu took out his mouse and swung it around as he walked among them.

To achieve your goal, you must know your goal. Once you have a better understanding of it, you can search for it more effectively.  That makes every search have two parts. The first is to find a dictionary, encyclopedia, or major site that has a "write up" for your subject.  That initial study will help to color every search made on that subject.

What are you searching for?

Keywords, important phrases, and "getting a feel" for what is important.  One must load the search with the appropriate key words and the appropriate order.  This will vary from search engine to search engine, just as striking motions vary from martial art to martial art.

The second is the actual "combing the web" to get the information you exactly want.

Selection and understanding of your Search Engine is important. But that is for another, more advanced class. No matter what engine you use, the following information will be useful.

You must learn how the search engines you are using sort information, their categories. (Check out some search responses.) That way you can enter keywords in an order that will be more effective.  For example Hotel, Palo Alto, CA will return different results and less useful ones if one wants a hotel in Palo Alto, CA, than Palo Alto, Ca, Hotel search in most engines.

Also having a feel for the subject is important. For example, if one looked though a map of Palo Alto (part of one initial study), one would realize that the location inside the city would be very important and that by expanding one's search to the nearby cities (one of which is only 120 yards/meters from "Downtown"), one could get a "better rate" for a "better room".

Once you have done the initial set of studies and searches, you can apply this information to any searches done down the line OR EVEN similar searches.  Thus you will learn which sites were the easiet for you to reference certain information (easy to use for you), so you know that you will like the responses on those sites.

Kata: You can practice doing RPG, Table Top RPG, Role-Playing, Historical Simulation, searches. After you have done a search or two in these areas, any future search - which will be for a specific topic - will be faster and more effective.

[MoonHunter]

The wind blew the cherry blossoms along the grounds. The rhythmic click, click of mousing exercises blew through the dojo and out to the wind. The Si-fu walked among the students observering their form when he began.

"Often time the best way to search is by using the right engine for the job. Searching is as an individual process as martial combat. Each person has just the right style for their personality and physicality. One might excel in Aikedo, others Tai Kwan Do, yet others might only excell in White Crane Kung-fu. Others might use Linux. Most people only use Yahoo! or Google or MSN or what ever engine is under the portal they visit. They assume that this "one style fit all" approach of most portals will be good for them. Sometimes they are right. Many times they are wrong. There are dozens of search engines out there.

The best one for you is based on two criteria:
1) Does the web search for the subject or area of the web you are interested in? You may need a specialty search engine rather than a general one. Also every site does not reach every part of the web, so each one will return different answers.

2) Does the search engine return results in a format that make sense to you? 2000 valid responses are useless if you can't understand them quickly.

What works well for me, might not work well for you. Google-fu is very JKD that way. Some people need breadth of search, others need depth, others need a 100% correspondence on the first hit, while others like to wade through a few sites before they focus on one. You need to find your own style of search, one that fits your needs, attention span, and reading speed.

I read fast and like spartan responses. I search dozens of pages before I decide to go back and investigate just one of them. (back linking to the seach page, so I have a continued choice). I use Google, Yahoo, Dmoz, Altavista, and a whole bunch of specialty sites that are not easy to use, but are useful for specific pieces of search.

One should play with a few different search engines to find the one you like. Do the same search on a few different sites (let's say RPG ), and see how the results differ.  So try them out. See what they have for you.

One good place to start is on the Search Engine Guide:
http://www.searchengineguide.com/searchengines.html

If you want a little more info, before you go diving in.

Google
http://www.google.com
Google has a well-deserved reputation as the top choice for those searching the web. The crawler-based service provides both comprehensive coverage of the web along with great relevancy. It's highly recommended as a first stop in your hunt for whatever you are looking for.

Google provides the option to find more than web pages, however. Using on the top of the search box on the Google home page, you can easily seek out images from across the web, discussions that are taking place on Usenet newsgroups, locate news information or perform product searching. Using the More link provides access to human-compiled information from the Open Directory (see below), catalog searching and other services.

Yahoo!
http://www.yahoo.com
Launched in 1994, Yahoo is the web's oldest "directory," a place where human editors organize web sites into categories. However, in October 2002, Yahoo made a giant shift to crawler-based listings for its main results. These came from Google until February 2004. Now, Yahoo uses its own search technology. Learn more in this recent review from our SearchDay newsletter, which also provides some updated submission details. In addition to excellent search results, you can use tabs above the search box on the Yahoo home page to seek images, Yellow Page listings or use Yahoo's excellent shopping search engine. Or visit the Yahoo Search home page, where even more specialized search options are offered.

The Yahoo Directory still survives. You'll notice "category" links below some of the sites lists in response to a keyword search. When offered, these will take you to a list of web sites that have been reviewed and approved by a human editor.


Ask Jeeves
http://www.askjeeves.com
Ask Jeeves initially gained fame in 1998 and 1999 as being the "natural language" search engine that let you search by asking questions and responded with what seemed to be the right answer to everything. In reality, technology wasn't what made Ask Jeeves perform so well. Behind the scenes, the company at one point had about 100 editors who monitored search logs. They then went out onto the web and located what seemed to be the best sites to match the most popular queries. Today, Ask Jeeves instead depends on crawler-based technology to provide results to its users. These results come from the Teoma search engine that it owns, which is described below.

AllTheWeb.com
http://www.alltheweb.com
Powered by Yahoo, you may find AllTheWeb a lighter, more customizable and pleasant "pure search" experience than you get at Yahoo itself. The focus is on web search, but news, picture, video, MP3 and FTP search are also offered.

AOL Search
http://aolsearch.aol.com (internal)
http://search.aol.com/(external)

AOL Search provides users with editorial listings that come Google's crawler-based index. Indeed, the same search on Google and AOL Search will come up with very similar matches. So, why would you use AOL Search? Primarily because you are an AOL user. The "internal" version of AOL Search provides links to content only available within the AOL online service. In this way, you can search AOL and the entire web at the same time. The "external" version lacks these links. Why wouldn't you use AOL Search? If you like Google, many of Google's features such as "cached" pages are not offered by AOL Search.

HotBot
http://www.hotbot.com
HotBot provides easy access to the web's three major crawler-based search engines: Yahoo, Google and Teoma. Unlike a meta search engine, it cannot blend the results from all of these crawlers together. Nevertheless, it's a fast, easy way to get different web search "opinions" in one place.

Teoma
http://www.teoma.com
Teoma is a crawler-based search engine owned by Ask Jeeves. It has a smaller index of the web than its rival crawler-competitors Google and Yahoo. However, being large doesn't make much of a difference when it comes to popular queries, and Teoma's won praise for its relevancy since it appeared in 2000.

AltaVista
http://www.altavista.com
AltaVista opened in December 1995 and for several years was the "Google" of its day, in terms of providing relevant results and having a loyal group of users that loved the service. This is still a key site because it will easily search for responses in other languages, and has a translation service.


Gigablast
http://www.gigablast.com
Compared to Google, Yahoo or even Teoma, Gigablast has a tiny index of the web. However, the service is constantly gaining new and interesting features. Give it a whirl, if you want to try something experimental yet dependable.

LookSmart
http://www.looksmart.com
LookSmart is primarily a human-compiled directory of web sites. It gathers its listings in two ways. Commercial sites pay to be listed in its commercial categories, making the service very much like an electronic "Yellow Pages." However, volunteer editors at the LookSmart-owned Zeal directory also catalog sites into non-commercial categories for free. Though Zeal is a separate web site, its listings are integrated into LookSmart's results.

Lycos
http://www.lycos.com
Lycos is one of the oldest search engines on the web, launched in 1994. It ceased crawling the web for its own listings in April 1999 and instead provides access to human-powered results from LookSmart for popular queries and crawler-based results from Yahoo for others.

MSN Search
http://search.msn.com
Formerly one of Search Engine Watch's top choices, MSN Search is definitely one to watch.

Netscape Search
http://search.netscape.com
Owned by AOL Time Warner, Netscape Search uses Google for its main listings, just as does AOL's other major search site, AOL Search. So why use Netscape Search rather than Google? Unlike with AOL Search, there's no compelling reason to consider it. The main difference between Netscape Search and Google is that Netscape Search will list some of Netscape's own content at the top of its results. Netscape also has a completely different look and feel than Google. If you like either of these reasons, then try Netscape Search. Otherwise, you're probably better off just searching at Google.

Open Directory
http://dmoz.org/
The Open Directory uses volunteer editors to catalog the web. While you can search at the Open Directory site itself, this is not recommended. The site has no "backup" results that kick in should there not be a match in the human-compiled database. In addition, the ranking of sites during keyword searching is poor, while alphabetical ordering is used when you choose to "browse" categories by topic.

Other things to look at
http://www.searchenginewatch.com/links/article.php/2156351
Specialty search engines. Really specific ones as well.

http://searchenginewatch.com/links/article.php/2156181
Invisible web. There is helpful information locked away in databases that can never be indexed by search engines. These sites find it.

Metacrawlers
These sites submit your request to several sites at once and pull back the first 10 or so responses from those pages. The results can be confusing and not always accurate...

Dogpile
http://www.dogpile.com
Popular metasearch site owned by InfoSpace that sends a search to a customizable list of search engines, directories and specialty search sites, then displays results from each search engine individually.

Vivisimo
http://vivisimo.com/
Enter a search term, and Vivismo will not only pull back matching responses from major search engines but also automatically organize the pages into categories. Slick and easy to use.

HotBot
http://www.hotbot.com
Technically HotBot isn't a meta crawler, in that it doesn't search several search engines all at once. However, it is unique in providing direct, dependable and easy access to the web's four major crawler-based search engines: Yahoo (use the HotBot button), AllTheWeb (use the Lycos button, and note that LookSmart results appear for more popular queries), Google and Teoma (use the Ask Jeeves button).

Kartoo
http://www.kartoo.com
If you like the idea of seeing your web results visually, this meta search site shows the results with sites being interconnected by keywords.

Mamma
http://www.mamma.com
Founded in 1996, Mamma.com is one of the oldest meta search engines on the web. Mamma searches against a variety of major crawlers, directories and specialty search sites. The service also provides a paid listings option for advertisers, Mamma Classifieds.

SurfWax
http://www.surfwax.com
Searches against major engines or provides those who open free accounts the ability to chose from a list of hundreds. Using the "SiteSnaps" feature, you can preview any page in the results and see where your terms appear in the document. Allows results or documents to be saved for future use.

Excite
http://www.excite.com
Formerly a crawled-based search engine, Excite was acquired by InfoSpace in 2002 and uses the same underlying technology as the other InfoSpace meta search engines, but maintains its own portal features.

Search Engine Watch
There is a site about nothign but search engines. Here is a listing of their most common search engines and their purposes.
http://www.searchenginewatch.com/links/index.php

[MoonHunter]

As the class continues to type away, searching for eternal answers, the Si-fu continues.

There are little things you can do to increase your speed and accuracy.

The first is to do a first page search. Your search engine will give you a list of 5 to 25 listings per page.  Click the link and scan through the first page to see if a) it has the information you want or b) a link to the page that might.  Unless the page seems "perfect" to you, hit the back button to the list. If the page seemed possibly promising, remember which one it was on the page. Check out all the pages on that first search page (or the ones with the promising blurbs). Once you have done that, go back and take a serious look at the pages that seemed promising. Once you have gone through those, go on to the next "page" of 5 to 25 links.

Ignore the paid links in most cases, unless you are trying to do commerace. Information will not be free in most paid links.

The old axiom of professional looking pages have professional information. The more effort someone has poured into a page, the more likely they have poured effort into information, or at least into the organization of the information. This is a rough rule, not a law.

If the page has something interesting, but not immediately helpful... copy the url (alt-d, ctrl-c) then press ctrl-n (or what ever creates a new window for your browser), then paste the url into that window (alt-d ctrl-v). Tab back to your original window (win-tab or alt-tab) and continue to search. The window will sit there for you to look at later.

Learn the keystroke shortcuts for your browser. The less time you spend messing with your mouse the more time you spend moving information. The mouse should be used for games, when using the keyboard will take too long (like tabbing down two dozen links), or as a last resort.

Feel free to try two or three sets of key words that might find the answer you want. If you can't get the information you want, you are probably searching the wrong key words.  

You can have several seach windows open at once. That way you can be doing something while an engine is searching for you.

A simple way to get other categories and keywords is to goto www.askjeeves.com and put your question in "natural English". It will put out a list of possibilities. You will usually find more effective keywords to search with other engines.

[MoonHunter]

The setting in a little restraunt down below the Google-Fu temple. The students and the staff have come to relax while the maintance crew does the floor.

After much etoh....

So I do like to check out webcams to watch the world too! http://www.rpgcitadel.com/guild/index.php?topic=2468.0

But you can see what the rest of the world searches too!

http://www.google.com/press/zeitgeist.html
What people are searching for at Google and its associated specialty services in a variety of categories. There are versions for various countries, as well.

http://www.imagine-msn.com/insider/
Top 200 queries on MSN Search (annoyingly in random order), top "movers" in TV, sports and music, and a "duels" feature pitting top queries in a race against each other.  You can also find a lite version of this on the main msn page.

http://buzz.yahoo.com/
Shows you what's hot and what's not in terms of search topics at Yahoo. There is also a lite list on the main page.

http://hot.aol.com/hot/hot
Top current queries, or see those in the last hour, last day and within particular categories.

There are some other sites that provide similar things. These are less admusing and popular culture, but provide insight on how engines work and sort:

dWoz Search Phrase Lists:
http://www.dwoz.com/default.asp?Pr=122
Great directory of services like those above, for search engines large and small, across the web. Also see the Search Spies & Voyuers category.
 
Yahoo Keyword Selector Tool:
http://inventory.overture.com/d/searchinventory/suggestion/
Formerly called the Overture Keyword Selector Tool and the Overture Search Term Suggestion Tool before that, this free service is primarily designed to help advertisers who wish to select terms to target with ads on the Yahoo network. But you can use it to see how popular particular terms are.
 
Google AdWords Keyword Tool:
https://adwords.google.com/select/main?cmd=KeywordSandbox
Similar to Overture's tool above, this is designed to help Google advertisers know what terms to target but can be used by anyone to gauge the popularity of terms.
 
And never forget about
http://searchenginewatch.com/

[MoonHunter]

Have you ever tried to find something on a site, but the search function was... less than up to the task?  There is a trick that will work to help you.

1) Goto www.google.com
2) Type site:www.the_site_you_want_to_search.com and then the search functions.
3) press enter

Everything on that site that matches will come up.

If you wanted read everything MoonHunter has ever posted on www.strolen.com (And who wouldn't?), you would type

site:www.strolen.com MoonHunter

Want to see every mention of Elves?
site:www.strolen.com Elf Elves Elventi Faries  
  (Remember extra search words will return extra results)

[MoonHunter]

http://www.google.com/webhp?complete=1&hl=en
As you type your search, Google offers keyword suggestions in real time

There are many Google Beta and odd features you might find useful.
http://labs.google.com/

Let us not forget about all the other things it does.
http://print.google.com/intl/en/options/

Now Yahoo has dozens of additional features as well, you just have to look harder to find them off the front page.

[Nosredna]

Hey guys I'm back.

Here's a pic I found.

[Rei]

WHAO!

umm... that was a bit extreme... ^_^""

[Styke_Mondshein_347]

Cool pic. Nice tips. Works nicely. Yeah!

[MoonHunter]

Does anyone have any specific questions about searching?

[MoonHunter]

Another shameless bump into the forums.  I would make this an article, but... it is not very game related.

[Strolen]

Moon has some great lists of alternate search engines. Here is a link to another article on "The Top 100 Alternative Search Engines."

Ask anyone which search engine they use to find information on the Internet and they will almost certainly reply: "Google." Look a little further, and market research shows that people actually use four main search engines for 99.99% of their searches: Google, Yahoo!, MSN, and Ask.com (in that order). But in my travels as a Search Engine Optimizer (SEO), I have discovered that in that .01% lies a vast multitude of the most innovative and creative search engines you have never seen. So many, in fact, that I have had to limit my list of the very best ones to a mere 100.

But it's not just the sheer number of them that makes them worthy of attention; each one of these search engines has that standard "About Us" link at the bottom of the homepage. I call it the "why we're better than Google" page. And after reading dozens and dozens of these pages, I have come to the conclusion that, taken as a whole, they are right!

http://www.readwriteweb.com/archives/top_100_alternative_search_engines.php

[MoonHunter]

The right tool for the right job and you have to make sure that the tool "fits your hand".  Thus experimenting with a couple of engines for types of searches you commonly do can produce a great deal of useful information.

Now I love these "why we are better than google pages". Many want to sport their "shiny new algerythm" or how they utilize other search engines to get the best responses. I especially like the, "in the future we won't do (as many or annoying) ads, but have to right now to make ends meet". comments that show up in these pages.

What I really wish these people would do is list what their searches are best at.

[MoonHunter]

Moon goes gets a broom and begins to sweep out the Dojo floor.  It has been a while since it was open. Lets us bring in the air.

[Kassy]

I worry as to your sanity with all this about google, but i like it anyhows.

[MoonHunter]

You should not. The internet has a vast sum of human knowledge. However, if you can't access it... it is like it is not there.

The term of google-fu came from the search engine community, as the skill to use a search algorythm to its maximum effect. I learned the term via slashdot. Being a long term computer user, I love search engines.  I mean long term. I mean TSR80, pre Apple II, Vic 20 kind of long term. Imagine looking for things on the internet without search engines.  You had a few key sites. They had links to maybe a few other sites. You had to read and read and follow links to find things. Utilizing a search engine to actually find something useful/ what you wanted, was like Nirvana. The net became useful.

Of course the first one was yahoo.com. They became portals (which were good in the begining). But yahoo-fu sounds funny. But we move on.

I remember when the Net was hard to use. I remember when it was not easy to access. If you think it is tough to find what you want now, you should of tried it back then. Back when we had to carry our punch cards... up hill.. both ways.. in the snow..  and modems were screaming at 1.2k baud.  And if you don't know what that 1.2k baud is...  it is what you would consider a stalled connection. 1/32th of what you connect to AOL at.. and 1/200ths of what you get when you connect by dsl or cable.

[Kassy]

Fair enough then, i won't argue, but i still think this one's a bit random.

[valadaar]

  1.2k baud? Luxury!

I've still got a 300 baud modem in a box somewhere....

[MoonHunter]

I started out with a Vic 20 because one of the two modems available for it was 1200 baud. It was a huge 75 dollars more (and when you are buying the computer for 99.95....)

[Pariah]

We got ourselves a high roller here gents...

[MoonHunter]

Pariah, this was long enough ago, that you have to

a) Adjust for inflation...

b) I was in High School and the minimum wage was 2.65 (though I was making more than that).

c) Remember that at the time your average personal computer costed 300 dollars.
--- and had the processing power of your average electronic toy now sold for under five bucks

[Pariah]

I know boss, I'm just messing with one of our resident fossils. 

[MoonHunter]

With a snap, the paper screen door slide open. The Sensei walked down the creaky wooden steps out to the green field towards several students practicing archery with their mice. The Sensei squinted and looked at the screens "down range".

Many of the students were having trouble hitting their targets.

"You are all aiming too tightly. With such limiting, you miss what you are looking for because it is not worded exactly as you are wording it. Sometimes to hit exactly what you need, you must look 'around the target', taking in the entire target. Only then, will you find 'the path' your search needs to reach its target."

Now a good way to do maximize your search results is to be fuzzy.

Try searching a couple of mispellings, it is amazing what will come up.

Goto reference.com and then to thesaurus.com or encyclopedia.com. Punch your keyword or desired search in, then get a ton of related things. These become new key words to increase your focus or as is usually the case, the breadth of your search.

You then put this information in. 
So instead of just seaching specific wine or wine related thing, you would search Madeira, oak, Champagne, vineyard, or some combination or
and get more hits, and possibly more refined hits.  From there, you can weed out what is good and what is bad.

Remember, speed reading is your best google-fu tool