Difficulty building a filtered search feature

Hollister11

Is it difficult to build a filtered search feature?

So for example
Done deal (Min & Max price, Min and Max Year, Color, Location)
Daft (Min & Max price, Min & Max Bedrooms, Available from, location)

Is this extremely difficult, or is it relatively straightforward, just time-consuming?

My idea for my final year project will require a search feature like this, so I'm looking for advice?

Idleater

Hollister11 wrote: »

Is it difficult to build a filtered search feature?

So for example
Done deal (Min & Max price, Min and Max Year, Color, Location)
Daft (Min & Max price, Min & Max Bedrooms, Available from, location)

Is this extremely difficult, or is it relatively straightforward, just time-consuming?

My idea for my final year project will require a search feature like this, so I'm looking for advice?

Do you mean search in general, or searching a 3rd party site/api?

If you're looking at 3rd party sites that don't have an api, have a read up on web scraping. Something like BeautifulSoup for python would be a candidate.

If you're in control of your data and want to leverage search on your own site/platform, then yes, it's relatively straight forward. One starting point for reading up is ORM, where you have some sort of framework that converts database rows to objects that you can manipulate and process for a front end.

Hollister11

Idleater wrote: »

Do you mean search in general, or searching a 3rd party site/api?

If you're looking at 3rd party sites that don't have an api, have a read up on web scraping. Something like BeautifulSoup for python would be a candidate.

If you're in control of your data and want to leverage search on your own site/platform, then yes, it's relatively straight forward. One starting point for reading up is ORM, where you have some sort of framework that converts database rows to objects that you can manipulate and process for a front end.

It would be a search using my own data.

No Web scrapping, just user submitted data.

Buford T Justice

To be fair, it really depends on the structure of your data. If its organised in a table(s) and each of the criteria have their own fields, then its just a case of building up a complex where statement really. I can't speak for noSql equivalents though

Hollister11

I just a feature in my project. I haven't put too much thought into it.

But it's going to be in MySQL

14ned

Hollister11 wrote: »

Is it difficult to build a filtered search feature?

So for example
Done deal (Min & Max price, Min and Max Year, Color, Location)
Daft (Min & Max price, Min & Max Bedrooms, Available from, location)

Is this extremely difficult, or is it relatively straightforward, just time-consuming?

My idea for my final year project will require a search feature like this, so I'm looking for advice?

Look into https://lunrjs.com/, it's a 100% client side Javascript site search. Awfully inefficient, but quick to hack in.

If you do it properly, you need to set up some SQL indexes on your SQL store which match the searchable things, then do a JOIN query on them. For an academic piece of work, this is the method they'll be expecting and not doing this will likely cost you in grade.

Niall

Talisman

It's easy to achieve what you are looking for using Apache Solr. Using a real search platform can facilitate facet searching (your filtered search) as well as other nice to have features such as spell checking, phonetic search and query completion. Essentially you would schedule Solr to index your database and then your queries run against the search index. You'll find plenty of tutorials online that will demonstrate how to configure Solr to do exactly what you want - there is no need to go near SQL unless you absolutely have to directly query the database.

John_Mc

Wow. All these replies and no one suggesting a simple SQL query. This is exactly what it's designed for.

Idleater

John_Mc wrote: »

Wow. All these replies and no one suggesting a simple SQL query. This is exactly what it's designed for.

search feature != a sql query.

John_Mc

Idleater wrote: »

search feature != a sql query.

Really? Although it's not the only component, it is the main component of a search feature.

A UI to capture the filter criteria, and the SQL usually applies the filtering using conditional clauses.

Idleater

John_Mc wrote: »

Really? Although it's not the only component, it is the main component of a search feature.

A UI to capture the filter criteria, and the SQL usually applies the filtering using conditional clauses.

And you reimplement a new sql query for each new search area?

I understand your point, and yes, at the end of the day, an sql query should be run, but the maintenance goes up significantly if you leverage individual sql queries per search type so to speak.

Maybe the OP only has to retrieve all records in a 1 table system and filter them in his/her code.

John_Mc

Idleater wrote: »

And you reimplement a new sql query for each new search area?

I understand your point, and yes, at the end of the day, an sql query should be run, but the maintenance goes up significantly if you leverage individual sql queries per search type so to speak.

Maybe the OP only has to retrieve all records in a 1 table system and filter them in his/her code.

It totally depends on the situation. Most applications only have a few search fields. Ideally your SQL query can be reused for all in the form of a stored procedure, but that's only possible if you're searching across the same set of data.

You yourself suggested an ORM which translates to a SQL statement. I use ORMs where possible but if the OP is asking about how to search like daft or done deal then they probably aren't at that level yet. You should be familiar with SQL before working with ORMs in my opinion.

jmcc

John_Mc wrote: »

Wow. All these replies and no one suggesting a simple SQL query. This is exactly what it's designed for.

SQL is fine when you have the data in a table or tables. When it is on the web, the data has to be collected and the ETLed first. The SQL aspect is easy enough. Collecting the data might be more problematic unless the sites in question have data/API feeds.

Regards...jmcc