I hope you'll like a tool I've made called Nautilus.
Nautilus (sqlserver) v1.0.1
Google your MS SQLServer database
http://sourceforge.net/projects/nautilus/
Author: Marcos Luis Casamayor (marcos.casamayor@.gmail.com)
WHAT
--
Nautilus is a tool for developers, advanced users and auditors.
The main objetive is to find information in a SQLServer database without knowing too much about the database. User can search tables and records with no SQL knowledge required.
You can also browse related data following the foreign key links.Since this appears to be a freeware application, I am going to let it stay in this forum. If I find out otherwise I will kick the thread.|||Product won't handle integrated security. Requires SQL Server login.
Unable to use the product because "Page cannot be displayed" errors in application.
Uninstalling...|||...I have a script that will do this anyway, if anybody needs this functionality.|||Product won't handle integrated security. Requires SQL Server login.
Unable to use the product because "Page cannot be displayed" errors in application.
Uninstalling...
Its true, the tool requires knowing database connection info.
But once loged in you can find tables (sometimes you dont remember exact name), then click over it and see its data and structure, then locate a record filtering data, then pivot a record, see its complete info and follow the foreign keys "paths" to the records it refers to and see which records refer it. A diferent feature is finding a value in any record of any table which only sometimes is needed (this is the "google data" option).
I know that if you give it a try you won't be disappointed.
Marcos.|||I did give it a try. All I got was "page cannot be displayed" errors.|||I did give it a try. All I got was "page cannot be displayed" errors.
blindman, I'm very sorry about your problems. I would like some clues about your environment (sqlserver version, xp version, and more).
I can only guess that you have c:\nautilus\nautilus\ in your disk and you created the virutal dir to c:\nautilus instead of c:\nautilus\nautilus. There is always a confusion making zip files. I'll explain it better in next readme file.
sorry again, marcos.|||If anybody else on the forum tried this software, I encourage them to report their experiences.
By the way, marcoscasa, are you sure you can legally use the verb "google" to advertise your application? Has the term passed into the public domain? It implies that there is some sort of business relationship between you and the search engine company, or that your application uses the same search algorithms as google.
Showing posts with label google. Show all posts
Showing posts with label google. Show all posts
Monday, March 19, 2012
Google style search
Is there a way to do a google style search in SQL.
For example if I have a search field and someone puts in:
safe car
It will automatically search like this
contains(*,'"safe*" AND "car*"')
Or
if they put in
"safe car"
it would search in SQL like contains(*,'"safe car"')
Or
if they put in
safe -car AND dog
it would search in sql like contains(*,'"safe*" AND "dog*" AND NOT
"car"')
etc.
I guess I am looking for a regular expression or a script that would
parse the input field and output the user's query in a SQL server
acceptable format.
Google does a strict phrase based query, so safe car is searched as "safe"
or "car"
However to do what you want you would do the following:
Create PROCEDURE SearchSQL (@.stringin varchar(200), @.BooleanType int= NULL)
AS
-- a @.boolean type of null means a phrase based search
-- a @.boolean type of 0 means a phrase based search
-- a @.boolean type of 1 means an OR type search
-- a @.boolean type of 2 means an AND type search
-- a @.boolean type of 3 means an OR wildcarded type search
DECLARE @.holdingString VarChar(2000)
DECLARE @.whitespace INT
DECLARE @.boolean VarChar(10)
--returning a syntax message if no search phrase is passed
IF LEN(@.stringin)=0
BEGIN
PRINT 'usage is SimpleSQLFTSSearch ''Your Search Phrase goes here'''
RETURN -1
END
SET @.boolean=case WHEN @.booleantype=1 THEN char(34)+' OR ' + char(34)
WHEN @.booleantype=2 THEN char(34)+' AND ' + char(34)
WHEN @.booleantype=3 THEN char(34)+' OR ' + char(34)
ELSE ' ' END
DECLARE @.counter INT
DECLARE @.posold int
DECLARE @.posnew int
SET @.holdingstring='SELECT * FROM authors AS a JOIN
CONTAINSTABLE(authors,*,'''+char(34)
SELECT @.whitespace=LEN(@.stringin) - LEN(replace(@.stringin,' ',''))
SELECT @.posold=0
SELECT @.posnew=Charindex(' ',@.stringin)
WHILE @.whitespace >=0
BEGIN
IF @.whitespace=0
BEGIN
if @.booleanType =3
begin
SELECT
@.holdingString=@.holdingString+SUBSTRING(@.stringin, @.posold+1,LEN(@.stringin)-@.
posold+1)+'*'+char(34)+char(39)+',200) AS t ON '
end
else
begin
SELECT
@.holdingString=@.holdingString+SUBSTRING(@.stringin, @.posold+1,LEN(@.stringin)-@.
posold+1)+char(34)+char(39)+',200) AS t ON '
end
print @.holdingString
END
ELSE
BEGIN
if @.booleantype=3
begin
SELECT @.holdingString = CASE WHEN LEN(SUBSTRING(@.stringin,@.posold+1,
@.posnew-@.posold-1))>0 THEN @.holdingString+SUBSTRING(@.stringin,@.posold+1,
@.posnew-@.posold-1)+'*'+@.boolean ELSE @.holdingstring END
SELECT @.posold=@.posnew, @.posnew=Charindex(' ',@.stringin, @.posold+1)
END
else
begin
SELECT @.holdingString = CASE WHEN LEN(SUBSTRING(@.stringin,@.posold+1,
@.posnew-@.posold-1))>0 THEN @.holdingString+SUBSTRING(@.stringin,@.posold+1,
@.posnew-@.posold-1)+@.boolean ELSE @.holdingstring END
SELECT @.posold=@.posnew, @.posnew=Charindex(' ',@.stringin, @.posold+1)
end
end
SELECT @.whitespace=@.whitespace-1
END
SELECT @.holdingString = @.holdingString + 't.[KEY]=a.au_id ORDER BY RANK
DESC'
PRINT @.holdingstring
EXEC(@.holdingstring)
RETURN @.@.rowcount
--Usage is:
DECLARE @.returncode int
EXEC @.returncode=SearchSQL2 'this is a test',3
PRINT @.returncode
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Chuck P" <pe@.temp.gov> wrote in message
news:MPG.1cbcaa2b92fc73ee98969b@.news.microsoft.com ...
> Is there a way to do a google style search in SQL.
> For example if I have a search field and someone puts in:
> safe car
> It will automatically search like this
> contains(*,'"safe*" AND "car*"')
> Or
> if they put in
> "safe car"
> it would search in SQL like contains(*,'"safe car"')
> Or
> if they put in
> safe -car AND dog
> it would search in sql like contains(*,'"safe*" AND "dog*" AND NOT
> "car"')
> etc.
> I guess I am looking for a regular expression or a script that would
> parse the input field and output the user's query in a SQL server
> acceptable format.
|||thanks, Hillary
but I wanted to have more of the Google features
like searching for quoted phrases and using not or -
I think it will be a long process
For example if I have a search field and someone puts in:
safe car
It will automatically search like this
contains(*,'"safe*" AND "car*"')
Or
if they put in
"safe car"
it would search in SQL like contains(*,'"safe car"')
Or
if they put in
safe -car AND dog
it would search in sql like contains(*,'"safe*" AND "dog*" AND NOT
"car"')
etc.
I guess I am looking for a regular expression or a script that would
parse the input field and output the user's query in a SQL server
acceptable format.
Google does a strict phrase based query, so safe car is searched as "safe"
or "car"
However to do what you want you would do the following:
Create PROCEDURE SearchSQL (@.stringin varchar(200), @.BooleanType int= NULL)
AS
-- a @.boolean type of null means a phrase based search
-- a @.boolean type of 0 means a phrase based search
-- a @.boolean type of 1 means an OR type search
-- a @.boolean type of 2 means an AND type search
-- a @.boolean type of 3 means an OR wildcarded type search
DECLARE @.holdingString VarChar(2000)
DECLARE @.whitespace INT
DECLARE @.boolean VarChar(10)
--returning a syntax message if no search phrase is passed
IF LEN(@.stringin)=0
BEGIN
PRINT 'usage is SimpleSQLFTSSearch ''Your Search Phrase goes here'''
RETURN -1
END
SET @.boolean=case WHEN @.booleantype=1 THEN char(34)+' OR ' + char(34)
WHEN @.booleantype=2 THEN char(34)+' AND ' + char(34)
WHEN @.booleantype=3 THEN char(34)+' OR ' + char(34)
ELSE ' ' END
DECLARE @.counter INT
DECLARE @.posold int
DECLARE @.posnew int
SET @.holdingstring='SELECT * FROM authors AS a JOIN
CONTAINSTABLE(authors,*,'''+char(34)
SELECT @.whitespace=LEN(@.stringin) - LEN(replace(@.stringin,' ',''))
SELECT @.posold=0
SELECT @.posnew=Charindex(' ',@.stringin)
WHILE @.whitespace >=0
BEGIN
IF @.whitespace=0
BEGIN
if @.booleanType =3
begin
SELECT
@.holdingString=@.holdingString+SUBSTRING(@.stringin, @.posold+1,LEN(@.stringin)-@.
posold+1)+'*'+char(34)+char(39)+',200) AS t ON '
end
else
begin
SELECT
@.holdingString=@.holdingString+SUBSTRING(@.stringin, @.posold+1,LEN(@.stringin)-@.
posold+1)+char(34)+char(39)+',200) AS t ON '
end
print @.holdingString
END
ELSE
BEGIN
if @.booleantype=3
begin
SELECT @.holdingString = CASE WHEN LEN(SUBSTRING(@.stringin,@.posold+1,
@.posnew-@.posold-1))>0 THEN @.holdingString+SUBSTRING(@.stringin,@.posold+1,
@.posnew-@.posold-1)+'*'+@.boolean ELSE @.holdingstring END
SELECT @.posold=@.posnew, @.posnew=Charindex(' ',@.stringin, @.posold+1)
END
else
begin
SELECT @.holdingString = CASE WHEN LEN(SUBSTRING(@.stringin,@.posold+1,
@.posnew-@.posold-1))>0 THEN @.holdingString+SUBSTRING(@.stringin,@.posold+1,
@.posnew-@.posold-1)+@.boolean ELSE @.holdingstring END
SELECT @.posold=@.posnew, @.posnew=Charindex(' ',@.stringin, @.posold+1)
end
end
SELECT @.whitespace=@.whitespace-1
END
SELECT @.holdingString = @.holdingString + 't.[KEY]=a.au_id ORDER BY RANK
DESC'
PRINT @.holdingstring
EXEC(@.holdingstring)
RETURN @.@.rowcount
--Usage is:
DECLARE @.returncode int
EXEC @.returncode=SearchSQL2 'this is a test',3
PRINT @.returncode
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Chuck P" <pe@.temp.gov> wrote in message
news:MPG.1cbcaa2b92fc73ee98969b@.news.microsoft.com ...
> Is there a way to do a google style search in SQL.
> For example if I have a search field and someone puts in:
> safe car
> It will automatically search like this
> contains(*,'"safe*" AND "car*"')
> Or
> if they put in
> "safe car"
> it would search in SQL like contains(*,'"safe car"')
> Or
> if they put in
> safe -car AND dog
> it would search in sql like contains(*,'"safe*" AND "dog*" AND NOT
> "car"')
> etc.
> I guess I am looking for a regular expression or a script that would
> parse the input field and output the user's query in a SQL server
> acceptable format.
|||thanks, Hillary
but I wanted to have more of the Google features
like searching for quoted phrases and using not or -
I think it will be a long process
Google Maps
Has anybody tried to display a Google Map inside a report using Reporting
Services?
Thank you,
Alain Quesnel
alainsansspam@.logiquel.comOn Jun 26, 9:43 pm, "Alain Quesnel" <alainsanss...@.logiquel.com>
wrote:
> Has anybody tried to display a Google Map inside a report using Reporting
> Services?
> Thank you,
> Alain Quesnel
> alainsanss...@.logiquel.com
You could add a textbox control to the report and give it a string
value and set it to a Google Maps URL (via: right-click the textbox
contol -> select Properties -> Navigation -> Jump to URL:). Hope this
is helpful.
Regards,
Enrique Martinez
Sr. Software Consultant|||I'm already doing that, thanks. It's fine for a pdf file or for viewing the
report online. But once it's printed, thank link is no longer useful. I was
hoping to do something similar to a Windows app or a web page that has a
Google map frame in it.
Thanks,
Alain Quesnel
alainsansspam@.logiquel.com
"EMartinez" <emartinez.pr1@.gmail.com> wrote in message
news:1183083791.675971.61680@.k79g2000hse.googlegroups.com...
> On Jun 26, 9:43 pm, "Alain Quesnel" <alainsanss...@.logiquel.com>
> wrote:
>> Has anybody tried to display a Google Map inside a report using Reporting
>> Services?
>> Thank you,
>> Alain Quesnel
>> alainsanss...@.logiquel.com
>
> You could add a textbox control to the report and give it a string
> value and set it to a Google Maps URL (via: right-click the textbox
> contol -> select Properties -> Navigation -> Jump to URL:). Hope this
> is helpful.
> Regards,
> Enrique Martinez
> Sr. Software Consultant
>
Services?
Thank you,
Alain Quesnel
alainsansspam@.logiquel.comOn Jun 26, 9:43 pm, "Alain Quesnel" <alainsanss...@.logiquel.com>
wrote:
> Has anybody tried to display a Google Map inside a report using Reporting
> Services?
> Thank you,
> Alain Quesnel
> alainsanss...@.logiquel.com
You could add a textbox control to the report and give it a string
value and set it to a Google Maps URL (via: right-click the textbox
contol -> select Properties -> Navigation -> Jump to URL:). Hope this
is helpful.
Regards,
Enrique Martinez
Sr. Software Consultant|||I'm already doing that, thanks. It's fine for a pdf file or for viewing the
report online. But once it's printed, thank link is no longer useful. I was
hoping to do something similar to a Windows app or a web page that has a
Google map frame in it.
Thanks,
Alain Quesnel
alainsansspam@.logiquel.com
"EMartinez" <emartinez.pr1@.gmail.com> wrote in message
news:1183083791.675971.61680@.k79g2000hse.googlegroups.com...
> On Jun 26, 9:43 pm, "Alain Quesnel" <alainsanss...@.logiquel.com>
> wrote:
>> Has anybody tried to display a Google Map inside a report using Reporting
>> Services?
>> Thank you,
>> Alain Quesnel
>> alainsanss...@.logiquel.com
>
> You could add a textbox control to the report and give it a string
> value and set it to a Google Maps URL (via: right-click the textbox
> contol -> select Properties -> Navigation -> Jump to URL:). Hope this
> is helpful.
> Regards,
> Enrique Martinez
> Sr. Software Consultant
>
GOOGLE Like search database
Hi,
I am planning to build a search engine which works like GOOGLE. Can anybody
send me a sample database ..
regards LaraHi
Have a look at http://www.databaseanswers.org/data_models/index.htm
or ask Google for their trade secrets.
Regards
--
Mike Epprecht, Microsoft SQL Server MVP
Zurich, Switzerland
IM: mike@.epprecht.net
MVP Program: http://www.microsoft.com/mvp
Blog: http://www.msmvps.com/epprecht/
"Lara" <aneeshattingal@.hotpop.com> wrote in message
news:%23eWNoOXTFHA.2560@.TK2MSFTNGP09.phx.gbl...
> Hi,
> I am planning to build a search engine which works like GOOGLE. Can
> anybody
> send me a sample database ..
> regards Lara
>|||g*d d*mn it why don't you search google|||I have never seen Google's infrastructure, but I doubt it resembles:
http://www.databaseanswers.org/data...rch_engine.htm.
Either way, to the original question, if any of us had designed a database
to rival Googles, or even to work like theirs, don't you think we would be
out driving in our shiny new Jaguars! Seriously, this is quite a question
:)
----
Louis Davidson - http://spaces.msn.com/members/drsql/
SQL Server MVP
"Mike Epprecht (SQL MVP)" <mike@.epprecht.net> wrote in message
news:ON9VcdZTFHA.612@.TK2MSFTNGP12.phx.gbl...
> Hi
> Have a look at http://www.databaseanswers.org/data_models/index.htm
> or ask Google for their trade secrets.
> Regards
> --
> Mike Epprecht, Microsoft SQL Server MVP
> Zurich, Switzerland
> IM: mike@.epprecht.net
> MVP Program: http://www.microsoft.com/mvp
> Blog: http://www.msmvps.com/epprecht/
> "Lara" <aneeshattingal@.hotpop.com> wrote in message
> news:%23eWNoOXTFHA.2560@.TK2MSFTNGP09.phx.gbl...
>|||You might be surprised. All their spiders do is scan web pages and extract
keywords which they slap into a database structure. The complicated piece
is their spidering logic on the application side, which has to parse HTML
tags, ignore certain words, calculate "weights" for various pages, follow
links from one page to the next, calculate references to a page, etc. I
wouldn't be surprised at all if their database structure was based on
something very simple like this, with some additional columns and/or tables
to store word occurence counts, link counts and other page ranking
information.
Like you said though, too bad we didn't think of it first! :)
"Louis Davidson" <dr_dontspamme_sql@.hotmail.com> wrote in message
news:u2CbUvoTFHA.3244@.TK2MSFTNGP15.phx.gbl...
>I have never seen Google's infrastructure, but I doubt it resembles:
>http://www.databaseanswers.org/data...rch_engine.htm.
> Either way, to the original question, if any of us had designed a database
> to rival Googles, or even to work like theirs, don't you think we would be
> out driving in our shiny new Jaguars! Seriously, this is quite a question
> :)
> --
> ----
--
> Louis Davidson - http://spaces.msn.com/members/drsql/
> SQL Server MVP
> "Mike Epprecht (SQL MVP)" <mike@.epprecht.net> wrote in message
> news:ON9VcdZTFHA.612@.TK2MSFTNGP12.phx.gbl...
>|||Michael, is on the right track, but Google's Architecture (see
http://www.computer.org/micro/mi2003/m2022.pdf) and search algorithm
(PageRank - see http://www.voelspriet2.nl/PageRank.pdf) is far more complex.
They do not use RDBMS (except some MySQL for minor internal support apps)
and they do not use Windows servers and the 15,000 clustered servers in the
first article, is now well over 100,000 and growing... Interesting, many
people had thought of and had done search engines, just not with their
patented PageRank algorithm and of course having your own OS and file system
and 100,000 servers and many of the top PhD's doesn't hurt as well :-).
However, even with the above said, there are still opportunities in search
that they do not handle well, i.e., Enterprise Search (or intranet search)
with combining structured search (RDBMS) and unstructured search (spiders &
html) along with local search (or desktop search) seems to me to be an
interesting possibility at this time...
Regards,
John
--
SQL Full Text Search Blog
http://spaces.msn.com/members/jtkane/
"Michael C#" <xyz@.abcdef.com> wrote in message
news:hjbde.18763$V02.5438@.fe08.lga...
> You might be surprised. All their spiders do is scan web pages and
extract
> keywords which they slap into a database structure. The complicated piece
> is their spidering logic on the application side, which has to parse HTML
> tags, ignore certain words, calculate "weights" for various pages, follow
> links from one page to the next, calculate references to a page, etc. I
> wouldn't be surprised at all if their database structure was based on
> something very simple like this, with some additional columns and/or
tables
> to store word occurence counts, link counts and other page ranking
> information.
> Like you said though, too bad we didn't think of it first! :)
> "Louis Davidson" <dr_dontspamme_sql@.hotmail.com> wrote in message
> news:u2CbUvoTFHA.3244@.TK2MSFTNGP15.phx.gbl...
database
be
question
> ----
--
>|||Finally an intellignet answer to an unanswerable question... just kidding al
l.
Great response(s) though !
"Candor Feg" wrote:
> g*d d*mn it why don't you search google
>
>|||Thanks for the link - now THAT is interesting!
"John Kane" <jt-kane@.comcast.net> wrote in message
news:eZ4RkVzTFHA.1152@.tk2msftngp13.phx.gbl...
> Michael, is on the right track, but Google's Architecture (see
> http://www.computer.org/micro/mi2003/m2022.pdf) and search algorithm
> (PageRank - see http://www.voelspriet2.nl/PageRank.pdf) is far more
> complex.
> They do not use RDBMS (except some MySQL for minor internal support apps)
> and they do not use Windows servers and the 15,000 clustered servers in
> the
> first article, is now well over 100,000 and growing... Interesting, many
> people had thought of and had done search engines, just not with their
> patented PageRank algorithm and of course having your own OS and file
> system
> and 100,000 servers and many of the top PhD's doesn't hurt as well :-).
> However, even with the above said, there are still opportunities in search
> that they do not handle well, i.e., Enterprise Search (or intranet search)
> with combining structured search (RDBMS) and unstructured search (spiders
> &
> html) along with local search (or desktop search) seems to me to be an
> interesting possibility at this time...
> Regards,
> John
> --
> SQL Full Text Search Blog
> http://spaces.msn.com/members/jtkane/
>
> "Michael C#" <xyz@.abcdef.com> wrote in message
> news:hjbde.18763$V02.5438@.fe08.lga...
> extract
> tables
> database
> be
> question
> --
>|||Michael,
Just curious, which link did you find interesting? (I have many others, from
my book research on this subject.)
What do you think of "Enterprise Search" or Intranet search?
Thanks,
John
--
SQL Full Text Search Blog
http://spaces.msn.com/members/jtkane/
"Michael C#" <xyz@.abcdef.com> wrote in message
news:S0Bde.22739$RP1.19555@.fe10.lga...
> Thanks for the link - now THAT is interesting!
> "John Kane" <jt-kane@.comcast.net> wrote in message
> news:eZ4RkVzTFHA.1152@.tk2msftngp13.phx.gbl...
apps)
search
search)
(spiders
HTML
follow
I
would
>
-
>|||I found the architecture link the most interesting, although the Spidering
search was very informative as well. I found it very interesting how they
set up and configured their servers, as well as how they actually store
data. There definitely could be some lessons to be learned in there...
(now if I can just get my boss to hire a couple hundred developers we're on
our way! :) ) I'm particularly surprised that they don't use an RDBMS? I'm
still a little unclear on the method they use to store their data -- other
than physically separating the indexes from the data, which is pretty common
practice in SQL Server to improve performance. Do you have any more links
describing their architecture in greater detail, or do I have to wait for
the book? <g>
I had known a little bit about the page ranking formula they used, but
mostly just broad generalities. It was
to see specific calculations
and formulas, and how they relate to the overall search engine scheme.
I like the enterprise search idea, but I can imagine a decent enterprise
spider and search system would require some *serious* resources to
implement. It also brings up certain aspects of privacy and security, since
it would logically - if not physically - centralize access to corporate
information. After all, they might not want Joe Schmoe in the mailroom
spidering accounting and payroll information for the top execs in the
corporation -- and they definitely don't want John Schmoe in the
competitor's mailroom pulling up corporate docs at whim.
"John Kane" <jt-kane@.comcast.net> wrote in message
news:elFuBbAUFHA.3352@.TK2MSFTNGP12.phx.gbl...
> Michael,
> Just curious, which link did you find interesting? (I have many others,
> from
> my book research on this subject.)
> What do you think of "Enterprise Search" or Intranet search?
> Thanks,
> John
> --
> SQL Full Text Search Blog
> http://spaces.msn.com/members/jtkane/
>
> "Michael C#" <xyz@.abcdef.com> wrote in message
> news:S0Bde.22739$RP1.19555@.fe10.lga...
> apps)
> search
> search)
> (spiders
> HTML
> follow
> I
> would
> -
>
I am planning to build a search engine which works like GOOGLE. Can anybody
send me a sample database ..
regards LaraHi
Have a look at http://www.databaseanswers.org/data_models/index.htm
or ask Google for their trade secrets.
Regards
--
Mike Epprecht, Microsoft SQL Server MVP
Zurich, Switzerland
IM: mike@.epprecht.net
MVP Program: http://www.microsoft.com/mvp
Blog: http://www.msmvps.com/epprecht/
"Lara" <aneeshattingal@.hotpop.com> wrote in message
news:%23eWNoOXTFHA.2560@.TK2MSFTNGP09.phx.gbl...
> Hi,
> I am planning to build a search engine which works like GOOGLE. Can
> anybody
> send me a sample database ..
> regards Lara
>|||g*d d*mn it why don't you search google|||I have never seen Google's infrastructure, but I doubt it resembles:
http://www.databaseanswers.org/data...rch_engine.htm.
Either way, to the original question, if any of us had designed a database
to rival Googles, or even to work like theirs, don't you think we would be
out driving in our shiny new Jaguars! Seriously, this is quite a question
:)
----
Louis Davidson - http://spaces.msn.com/members/drsql/
SQL Server MVP
"Mike Epprecht (SQL MVP)" <mike@.epprecht.net> wrote in message
news:ON9VcdZTFHA.612@.TK2MSFTNGP12.phx.gbl...
> Hi
> Have a look at http://www.databaseanswers.org/data_models/index.htm
> or ask Google for their trade secrets.
> Regards
> --
> Mike Epprecht, Microsoft SQL Server MVP
> Zurich, Switzerland
> IM: mike@.epprecht.net
> MVP Program: http://www.microsoft.com/mvp
> Blog: http://www.msmvps.com/epprecht/
> "Lara" <aneeshattingal@.hotpop.com> wrote in message
> news:%23eWNoOXTFHA.2560@.TK2MSFTNGP09.phx.gbl...
>|||You might be surprised. All their spiders do is scan web pages and extract
keywords which they slap into a database structure. The complicated piece
is their spidering logic on the application side, which has to parse HTML
tags, ignore certain words, calculate "weights" for various pages, follow
links from one page to the next, calculate references to a page, etc. I
wouldn't be surprised at all if their database structure was based on
something very simple like this, with some additional columns and/or tables
to store word occurence counts, link counts and other page ranking
information.
Like you said though, too bad we didn't think of it first! :)
"Louis Davidson" <dr_dontspamme_sql@.hotmail.com> wrote in message
news:u2CbUvoTFHA.3244@.TK2MSFTNGP15.phx.gbl...
>I have never seen Google's infrastructure, but I doubt it resembles:
>http://www.databaseanswers.org/data...rch_engine.htm.
> Either way, to the original question, if any of us had designed a database
> to rival Googles, or even to work like theirs, don't you think we would be
> out driving in our shiny new Jaguars! Seriously, this is quite a question
> :)
> --
> ----
--
> Louis Davidson - http://spaces.msn.com/members/drsql/
> SQL Server MVP
> "Mike Epprecht (SQL MVP)" <mike@.epprecht.net> wrote in message
> news:ON9VcdZTFHA.612@.TK2MSFTNGP12.phx.gbl...
>|||Michael, is on the right track, but Google's Architecture (see
http://www.computer.org/micro/mi2003/m2022.pdf) and search algorithm
(PageRank - see http://www.voelspriet2.nl/PageRank.pdf) is far more complex.
They do not use RDBMS (except some MySQL for minor internal support apps)
and they do not use Windows servers and the 15,000 clustered servers in the
first article, is now well over 100,000 and growing... Interesting, many
people had thought of and had done search engines, just not with their
patented PageRank algorithm and of course having your own OS and file system
and 100,000 servers and many of the top PhD's doesn't hurt as well :-).
However, even with the above said, there are still opportunities in search
that they do not handle well, i.e., Enterprise Search (or intranet search)
with combining structured search (RDBMS) and unstructured search (spiders &
html) along with local search (or desktop search) seems to me to be an
interesting possibility at this time...
Regards,
John
--
SQL Full Text Search Blog
http://spaces.msn.com/members/jtkane/
"Michael C#" <xyz@.abcdef.com> wrote in message
news:hjbde.18763$V02.5438@.fe08.lga...
> You might be surprised. All their spiders do is scan web pages and
extract
> keywords which they slap into a database structure. The complicated piece
> is their spidering logic on the application side, which has to parse HTML
> tags, ignore certain words, calculate "weights" for various pages, follow
> links from one page to the next, calculate references to a page, etc. I
> wouldn't be surprised at all if their database structure was based on
> something very simple like this, with some additional columns and/or
tables
> to store word occurence counts, link counts and other page ranking
> information.
> Like you said though, too bad we didn't think of it first! :)
> "Louis Davidson" <dr_dontspamme_sql@.hotmail.com> wrote in message
> news:u2CbUvoTFHA.3244@.TK2MSFTNGP15.phx.gbl...
database
be
question
> ----
--
>|||Finally an intellignet answer to an unanswerable question... just kidding al
l.
Great response(s) though !
"Candor Feg" wrote:
> g*d d*mn it why don't you search google
>
>|||Thanks for the link - now THAT is interesting!
"John Kane" <jt-kane@.comcast.net> wrote in message
news:eZ4RkVzTFHA.1152@.tk2msftngp13.phx.gbl...
> Michael, is on the right track, but Google's Architecture (see
> http://www.computer.org/micro/mi2003/m2022.pdf) and search algorithm
> (PageRank - see http://www.voelspriet2.nl/PageRank.pdf) is far more
> complex.
> They do not use RDBMS (except some MySQL for minor internal support apps)
> and they do not use Windows servers and the 15,000 clustered servers in
> the
> first article, is now well over 100,000 and growing... Interesting, many
> people had thought of and had done search engines, just not with their
> patented PageRank algorithm and of course having your own OS and file
> system
> and 100,000 servers and many of the top PhD's doesn't hurt as well :-).
> However, even with the above said, there are still opportunities in search
> that they do not handle well, i.e., Enterprise Search (or intranet search)
> with combining structured search (RDBMS) and unstructured search (spiders
> &
> html) along with local search (or desktop search) seems to me to be an
> interesting possibility at this time...
> Regards,
> John
> --
> SQL Full Text Search Blog
> http://spaces.msn.com/members/jtkane/
>
> "Michael C#" <xyz@.abcdef.com> wrote in message
> news:hjbde.18763$V02.5438@.fe08.lga...
> extract
> tables
> database
> be
> question
> --
>|||Michael,
Just curious, which link did you find interesting? (I have many others, from
my book research on this subject.)
What do you think of "Enterprise Search" or Intranet search?
Thanks,
John
--
SQL Full Text Search Blog
http://spaces.msn.com/members/jtkane/
"Michael C#" <xyz@.abcdef.com> wrote in message
news:S0Bde.22739$RP1.19555@.fe10.lga...
> Thanks for the link - now THAT is interesting!
> "John Kane" <jt-kane@.comcast.net> wrote in message
> news:eZ4RkVzTFHA.1152@.tk2msftngp13.phx.gbl...
apps)
search
search)
(spiders
HTML
follow
I
would
>
-
>|||I found the architecture link the most interesting, although the Spidering
search was very informative as well. I found it very interesting how they
set up and configured their servers, as well as how they actually store
data. There definitely could be some lessons to be learned in there...
(now if I can just get my boss to hire a couple hundred developers we're on
our way! :) ) I'm particularly surprised that they don't use an RDBMS? I'm
still a little unclear on the method they use to store their data -- other
than physically separating the indexes from the data, which is pretty common
practice in SQL Server to improve performance. Do you have any more links
describing their architecture in greater detail, or do I have to wait for
the book? <g>
I had known a little bit about the page ranking formula they used, but
mostly just broad generalities. It was
to see specific calculationsand formulas, and how they relate to the overall search engine scheme.
I like the enterprise search idea, but I can imagine a decent enterprise
spider and search system would require some *serious* resources to
implement. It also brings up certain aspects of privacy and security, since
it would logically - if not physically - centralize access to corporate
information. After all, they might not want Joe Schmoe in the mailroom
spidering accounting and payroll information for the top execs in the
corporation -- and they definitely don't want John Schmoe in the
competitor's mailroom pulling up corporate docs at whim.
"John Kane" <jt-kane@.comcast.net> wrote in message
news:elFuBbAUFHA.3352@.TK2MSFTNGP12.phx.gbl...
> Michael,
> Just curious, which link did you find interesting? (I have many others,
> from
> my book research on this subject.)
> What do you think of "Enterprise Search" or Intranet search?
> Thanks,
> John
> --
> SQL Full Text Search Blog
> http://spaces.msn.com/members/jtkane/
>
> "Michael C#" <xyz@.abcdef.com> wrote in message
> news:S0Bde.22739$RP1.19555@.fe10.lga...
> apps)
> search
> search)
> (spiders
> HTML
> follow
> I
> would
> -
>
Google Like Full Text Search
Hello,
I have a Full-Text Catalog that is populated by various columns in a few different tables. I have been able to create a stored procedure that will search across all of the different full-text columns and return me the results.
My problem is, if the someone searches for
hello world
then to my understanding I want to use FREETEXTTABLE to return my results (I actually get 0 results if I use CONTAINSTABLE)
If someone searches for
"hello world"
then I want to use CONTAINSTABLE because FREETEXTTABLE returns too many results.
And now the biggest problem would be, if someone searches for
"hello world" program
I would somehow need to use CONTAINSTABLE for the phrase and FREETEXTTABLE for 'program'. The SQL to accomplish something like this would probably be very ugly (if possible at all)
Can anyone give me any suggestions on this matter? I'm trying to create a google like search on a database which contains text and files (as BLOBs).
Thanks in advanceI should clarify a bit more.
If someone searches for
"hello world"
then I want to use CONTAINSTABLE because FREETEXTTABLE returns too many results.
The problem isn't that FREETEXTTABLE returns too many results but rather the rankings that I get back aren't helpful. Say I have 2 completely different tables and I want to search across both of them. Table1 has rows that contain the phrase "hello world" while Table2 doesn't contain that specific phrase, but does contain the separate words 'hello' and 'world'. I run FREETEXTTABLE on both Table1 and Table2, Union the results, and then return the table to my web app (where other formatting occurs before the results are displayed to the user.) Unfortunately because of how the Ranks are calculated, the entries in Table2 have a higher Rank than the entries in Table1 (even though the exact phrase occurs in Table1)
I want the rows from Table1 to appear before Table2 in this case.
One solution that I have been playing with is to always use CONTAINSTABLE, use regular expressions to insert 'AND' where applicaple, and use the 'FORMSOF(INFLECTIONAL, @.searchStr)' option. Unfortunately this seems to break when someone searches for
hello world
because I would insert an 'AND' between the words and then try to find the inflectional forms of "hello AND world" which of course breaks.
Once again, and ideas/suggestions would be greatly appreciated|||I think I've found the solution
If someone wants to search for...
"hello world" program
Then the sql would look like this...
SELECT *
FROM CONTAINSTABLE(T_Table, *, 'FORMSOF(INFLECTIONAL, "hello world") AND FORMSOF(INFLECTIONAL, "program")')|||Thanks EvilMonkey - very helpful. =)|||
cool but can u give some details.
Google DB Engine ?
Hi,
I was so lost in Google Desktop Search last night, I kept on dreaming
about the endless possiblisties of Google Search Technology.
While I was in school I had read enough about DB information
retrivial. Why can't google use their search technology to build
Database engine ? Isn't that a possiblity ?
Am I still dreaming or am I awake ?
Cheers,
da_LazyBoySearch is of course only part of the technology required for a database.
Also, search of unstructured text in web pages is quite different to data
retrieval in an RDBMS and RDBMS still dominates the commercial market... But
who knows? Give them 18 months...
--
David Portas
SQL Server MVP
--|||"David Portas" <REMOVE_BEFORE_REPLYING_dportas@.acm.org> wrote in message news:<z5SdnWvUFcMy0OzcRVn-tA@.giganews.com>...
> Search is of course only part of the technology required for a database.
> Also, search of unstructured text in web pages is quite different to data
> retrieval in an RDBMS and RDBMS still dominates the commercial market...
But who knows? Give them 18 months...
Exactly...Maybe even shorter time...Millions of Dollars are at stake
if they can do something...|||hello guys
there are a lot of fundamental differences between a Search engine
and a database. Both are very much distinct from one another. So can
google really build up a database of its own? thats a yes-no sort of
question.
If things would have been such easy and it would have ben possible
for a search engine developer to construct an RDBMS and vice-versa
then guys! Microsoft and Oracle would have each created some sort of
fastest search engines and linked them to their company web sites.
Developing a database requires years of skilled expertise in not only
the database technologies but also the age old database concepts which
are still dominant today.
So friends at last it stands that 'This is not an easy task(even in
18 months)'
not only for google but also for any other IT-giants.
Regards
Debashish|||We are just talking of a DB engine and not the DB itself. Just a component of it :)
Cheers
da_LazyBoy
debashish_majumdar@.rediffmail.com (debashish) wrote in message news:<bb05cfc5.0410170336.4899957e@.posting.google.com>...
> hello guys
> there are a lot of fundamental differences between a Search engine
> and a database. Both are very much distinct from one another. So can
> google really build up a database of its own? thats a yes-no sort of
> question.
> If things would have been such easy and it would have ben possible
> for a search engine developer to construct an RDBMS and vice-versa
> then guys! Microsoft and Oracle would have each created some sort of
> fastest search engines and linked them to their company web sites.
> Developing a database requires years of skilled expertise in not only
> the database technologies but also the age old database concepts which
> are still dominant today.
> So friends at last it stands that 'This is not an easy task(even in
> 18 months)'
> not only for google but also for any other IT-giants.
> Regards
> Debashish|||Take a look at Google's architecture here:
http://faculty.uwb.edu/aberger/CSS4...%20architecture.
pdf
*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!
Subscribe to:
Posts (Atom)