google showing LPSG.com

[Deno]

Several times now when I am searching for something google shows LPSG.com in the results. And displays posts that include words relating to the search topic. What makes a search engine be able to enter a site like this and find posts related to a search request.

 

[mindseye]

Several times now when I am searching for something google shows LPSG.com in the results. And displays posts that include words relating to the search topic. What makes a search engine be able to enter a site like this and find posts related to a search request.

Unregistered "guests" can read the messages on the site, but can't post messages. Google's bot often shows up here as a guest. Way back when LPSG was using YaBB (roughly 2003-2004 -- before InvisionBoard, which was before vBulletin, which we're using now), I blocked Google, MSN, and other search engines from reading potentially sensitive threads. This blocking was removed when we changed over to InvisionBoard.

 

[CarryMyJock]

Unregistered "guests" can read the messages on the site, but can't post messages. Google's bot often shows up here as a guest. Way back when LPSG was using YaBB (roughly 2003-2004 -- before InvisionBoard, which was before vBulletin, which we're using now), I blocked Google, MSN, and other search engines from reading potentially sensitive threads. This blocking was removed when we changed over to InvisionBoard.

Interesting ... forgive me for bumping up an old thread, but I'd be curious to know where things stand now, 1.5 yrs later - in regards to Google's bots .... ? I think it was very smart to block 'em due to sensitive threads.

 

[B_Think_Kink]

Interesting ... forgive me for bumping up an old thread, but I'd be curious to know where things stand now, 1.5 yrs later - in regards to Google's bots .... ? I think it was very smart to block 'em due to sensitive threads.

In the internet world sweety there is no such thing as sensitive topics. If you think there is, then you have a lot of desensitization to do. I think it would be nearly impossible to block bots, besides how would people find the place and become paying members if they couldn't find it?

 

[Mr. Snakey]

Just doing a normal search(Not google) a link to the gallery comes up again. I have no problem with this or anything else one can read.

 

[Mem]

I think that it is cool that if I Google my screen name I get a result with a link to here.

This is a public forum that is open for anyone to view, so I do not see a problem with a result coming up on Google.

 

[Mr. Snakey]

I think that it is cool that if I Google my screen name I get a result with a link to here.

This is a public forum that is open for anyone to view, so I do not see a problem with a result coming up on Google.

No big deal. They can see and read things but cant post without becoming a member.

 

[SteveHd]

CarryMyJock, no problem bumping the thread. The topic is still relevant.

The google/yahoo indexing of the site can be used to our advantage. I sometimes use them when I need to an "and" type of search.

 

[B_NineInchCock_160IQ]

I sometimes use Google to search for old threads because I find their search works better than LPSG's built-in one. Everyone should be aware that *everything* you post online is public information.

 

[CarryMyJock]

In the internet world sweety there is no such thing as sensitive topics. If you think there is, then you have a lot of desensitization to do. I think it would be nearly impossible to block bots, besides how would people find the place and become paying members if they couldn't find it?

OK, I just discovered an interesting gender disparity, in the way I reacted to you calling me "sweety". I first read this post on my phone, and did not see the name or gender of the poster, and I presumed male (because, in fact, the majority of members here are male).

I was immediately ticked off to be handed such condescension, when I was bringing to the forefront a very important issue. But then, when I saw the poster was a chick - somehow I wasn't ticked off at all - funny how context can change things, eh?

However, sweetie-poo, I must disagree with you in regards to "no such things as sensitive topics". HALF THE POINT OF THE INTERNET IS THE ABILITY TO HAVE ANONYMITY - and therefore not fear retribution for one thing or another, or for self-discovery. And, as the media has shown us very clearly lately - there's a large gap between what various cultures consider to be lewd. Did you hear about the rape victim that got punished with lashes for speaking to the media? I believe it was Saudi Arabia ... if a Saudi husband found his wife on this site -do you not see the potential for privacy concerns?

If nothing should be considered "sensitive" on the net, then why do we all have aliases and options about how much of our identity we reveal in our profiles?

See how the issue isn't skin deep here?

In fact, here's a perfect example of why the internet serves a great service of privacy to some: ShopInPrivate.com - Shop In Private At The World's Most Private Drugstore

As far as bots go - yes, there ARE ways to block bots. Have you ever been to webarchive.org and been told archiving has been blocked by "robots.txt" ?

And as far as finding the website go - there's a difference between being able to search the entire site - and its entire index of messages, or just being able to find a "Large Penis Support Group". Make sense ?:wink:

 

[CarryMyJock]

I sometimes use Google to search for old threads because I find their search works better than LPSG's built-in one. Everyone should be aware that *everything* you post online is public information.

EXACTLY - that's why I bumped up the thread. To remind people of that = and get a more recent clarification on this sites policies on search engines and bots.

 

[No_Strings]

HALF THE POINT OF THE INTERNET IS THE ABILITY TO HAVE ANONYMITY

Well, no, it isn't.

 

[CarryMyJock]

Well, no, it isn't.

OK, so are you just trying to be difficult, or do you really want to open up a discussion on the linquistic symantics of my statement?

When you looked at your options on how to "read" that sentance ... LITERALLY, wasn't the only one.

I think the general gist of the sentiment is clear, don't you?

 

[ManlyBanisters]

I think the general gist of the sentiment is clear, don't you?

Well, no, I don't

 

[SteveHd]

As far as bots go - yes, there ARE ways to block bots. Have you ever been to webarchive.org and been told archiving has been blocked by "robots.txt" ?

"Robots.txt" is based on an honor system. I'd guess google and yahoo do honor it, if present.

 

[No_Strings]

Well, no, I don't

:spankme:

 

[ManlyBanisters]

:spankme:

:biggrin1: I needed that...

Hey.. nobody said stop!

 

[CarryMyJock]

The spirit of me bumping this thread in the first place - was to INFORM. So I'm going to continue to do that, to those who will find the information useful and enlightening.

[FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif]Some sites are not available because of robots.txt or other exclusions. What does that mean?

[/FONT][/FONT][/FONT][/FONT][/FONT][/FONT] [FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][/FONT][/FONT][/FONT][/FONT][/FONT][FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif]The Standard for Robot Exclusion (SRE) is a means by which web site owners can instruct automated systems not to crawl their sites. Web site owners can specify files or directories that are disallowed from a crawl, and they can even create specific rules for different automated crawlers. All of this information is contained in a file called robots.txt. While robots.txt has been adopted as the universal standard for robot exclusion, compliance with robots.txt is strictly voluntary. In fact most web sites do not have a robots.txt file, and many web crawlers are not programmed to obey the instructions anyway. However, Alexa Internet, the company that crawls the web for the Internet Archive, does respect robots.txt instructions, and even does so retroactively. If a web site owner decides he / she prefers not to have a web crawler visiting his / her files and sets up robots.txt on the site, the Alexa crawlers will stop visiting those files and will make unavailable all files previously gathered from that site. This means that sometimes, while using the Internet Archive Wayback Machine, you may find a site that is unavailable due to robots.txt (you will see a "robots.txt query exclusion error" message). Sometimes a web site owner will contact us directly and ask us to stop crawling or archiving a site, and we endevor to comply with these requests. When you come across a "blocked site error" message, that means that a siteowner has made such a request and it has been honored.[/FONT][/FONT][/FONT][/FONT][/FONT][/FONT]

(yes, compliance with robots.txt is voluntary, but large SE's comply, in part due to corporate privacy /espionage issues)

[FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif]Is there any personal information in these collections?

[/FONT][/FONT][/FONT][/FONT][/FONT][/FONT] [FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][/FONT][/FONT][/FONT][/FONT][/FONT][FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif]We collect Web pages that are publicly accessible. These may include pages with personal information.[/FONT][/FONT][/FONT][/FONT][/FONT][/FONT]
[FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif][/FONT][/FONT][/FONT][/FONT][/FONT][/FONT]
[FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][/FONT][/FONT][/FONT][/FONT][/FONT][FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif]Who has access to the collections? What about the public?[/FONT][/FONT][/FONT][/FONT][/FONT][/FONT]
[FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif][/FONT][/FONT][/FONT][/FONT][/FONT][/FONT]
[FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][/FONT][/FONT][/FONT][/FONT][/FONT][FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif]Anyone can access our collections through our website archive.org. The web archive can be searched using the Wayback Machine. [/FONT][/FONT][/FONT][/FONT][/FONT][/FONT]
[FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif]The Archive makes the collections available at no cost to researchers, historians, and scholars. At present, it takes someone with a certain level of technical knowledge to access collections in a way other than our website, but there is no requirement that a user be affiliated with any particular organization.[/FONT][/FONT][/FONT][/FONT][/FONT][/FONT]


[FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif]How do you protect my privacy if you archive my site?[/FONT][/FONT][/FONT][/FONT][/FONT][/FONT] [FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][/FONT][/FONT][/FONT][/FONT][/FONT]
[FONT=Arial, Helvetica, sans-serif][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial,][FONT=Arial, Helvetica, sans-serif]The Archive collects Web pages that are publicly available � the same ones that you might find as you surfed around the Web. We do not archive pages that require a password to access, pages tagged for "robot exclusion" by their owners, pages that are only accessible when a person types into and sends a form, or pages on secure servers. We also provide information on removing a site from the collections. Those who use the collections must agree to certain terms of use.

Like a public library, the Archive provides free and open access to its collections to researchers, historians, and scholars. Our cultural norms have long promoted access to documents that were, but no longer are, publicly accessible.

Given the rate at which the Internet is changing � the average life of a Web page is only 77 days � if no effort is made to preserve it, it will be entirely and irretrievably lost. Rather than let this moment slip by, we are proceeding with documenting the growth and content of the Internet, using libraries as our model.

If you are interested in these issues, please join and contribute to our announcement and discussion lists.[/FONT][/FONT][/FONT][/FONT][/FONT][/FONT]