How to store data..

krazyklown

Just wondering what is the best way to store data for small utilities. Had to do a program for my sisters shop to basically search for products. Used a text file and created gui using visual express C++.
I want to do some similar small scale stuff so just wondering what is the best way to go about it. Are XML files the way to go or is there an alternative?
I think a database might be a bit overkill or should i be thinking along those lines?
Any input would be appreciated (in particular industry standard approach).
Thanks.

Sheeps

XML is good I suppose. I guess it depends on what kind of system you want to implement for her. Are there a lot of products?

dvpower

XML will be fine for small amounts of data. It was designed to be a transport mechanism, not a storage mechanism, but if your considering stroring data in a standard text file, then at least XML puts some structure on it.

It has a bad press (probably because people used to use it for stuff that outgrew it), but consider a MS Access database (esp is you own a copy of Access). You'll get good integration with Visual Studio; you'll be able to drag tables from the database explorer onto forms and have Visual Studio generate the data access code and bits of the UI for you.

Manach

At work we use a mixture of XML and CSV for short term data storage (5-1000 rows) & manipulation, but for enterprise level data storage and processing it goes into an Oracle DB.

fasty

SQLite would be a nice alternate to flat text files or XML.

sheesh

there is/was a small db application called cloudscape created by ibm and then handed over to opensource.

for web most places would use mysql or sqlserver.

krazyklown

Thanks for the replies guys. I just thought that maybe using text files would be amateurish but it seems not. Have used MYSQL for a website with PHP but not confident enough to try an integrate it with a GUI or anything like that.
At the risk of sounding stupid, lets say if i did use a DB, in order for the program to be portable wouldnt the database software have to be installed on the end users pc?
Lets say the likes of Outlook - how would it store contacts?

ressem

There's a compact version of SQL server for embedding within applications. Would be tidier than MySQL.

http://www.microsoft.com/sqlserver/en/us/editions/compact.aspx

If you use the embedded databases like this or sqllite, they are created/ copied with the application. Bit like outlook's PST files etc.

croo

If I was deciding, I would ask myself what kind of concurrency is required? Because this is the real benefit of a database server... to allow multiple read & potentially updates at the same time. If this app would need to manage such multiple concurrent access then the easiest route is a DB server of some sort, otherwise I wouldn't bother. Depending on the data it might be, that because of complexity or size, a DB might improve performance but I would wait until performance was an issue before I addressed that... if you design the app using the right patterns then the switch from XML files to a Db should be not be any great chore.

Naikon

Prolly go with XML as the structure is quite strict. Plenty of tools out there which can parse XML with ease. You could parse plain text too, but you would need a method of delimating text fields and manipulating tokens - not too trivial.

fergalr

I recommend sqllite.
Its easy to use, and more importantly, its got a nice migration path to a database if you need to change later.

If the project gets bigger than you anticipate, being able to switch to a database might be beneficial - or if you get a requirement for logging or backup or concurrency you didn't anticipate.

jimmycrackcorm

If there is any possibility that the applicaiton will grow in size and functionality then you need to start off with something more structured than text / xml.

krazyklown

Hiya no its just a mini project i have in mind. I am kinda hoping to break into IT so i am just trying to be as practical as i can even if its only for minor stuff. Thanks for the replies guys, much appreciated!

Evil Phil

I would go with a DB regardless of how small the application is, even if it's just a locally stored mdb file. Reasons being:

1. For dynamic data, such as a product list, a database will give you better performance and will be easier to maintain.
2. If you're trying to break into I.T., and I'm guessing you're talking about development specifically, then you'll need DB exposure.
3. You could still use XML for metadata, that is data that doesn't really change during the lifetime of the application.

Also, is C++ the best option for this. C# might get you to releasing a production version of your software a lot quicker. There's also more of a job market for it.

Zombrex

krazyklown wrote: »

Just wondering what is the best way to store data for small utilities. Had to do a program for my sisters shop to basically search for products. Used a text file and created gui using visual express C++.
I want to do some similar small scale stuff so just wondering what is the best way to go about it. Are XML files the way to go or is there an alternative?
I think a database might be a bit overkill or should i be thinking along those lines?
Any input would be appreciated (in particular industry standard approach).
Thanks.

To be honest I think a GUI in visual express accessing XML files is over kill.

You will end up writing your own basic database, why not just use an open source one like MySQL or a free one like SQL Server Express. You get a much better system that you would write yourself and you have all the advantages of a proper database.

If/When you want extra features as you feel you need them they are already there, you don't have to add them your GUI (the first thing would probably be backups)

MySQL server comes with third party GUIs but and SQL Server Express comes with one built it. You can get SQL Server Express up and running in a few minutes.

http://www.microsoft.com/express/database/

BostonB

I can't see any reason to use anything other than database and C#.

Enda_of_Eire

BostonB wrote: »

I can't see any reason to use anything other than database and C#.

.NET Reflector would be one of many reasons.
Unless ofcourse he wants to write open source projects.

Not to mention using these small fun projects as motivation to
write in a harder language like C++ and do you're own database like implementation,
which improves you're general programming skills
is probably another good reason.

<mod snip>

BostonB

This is a specific question about data storage .NET Reflector is not an answer to that question. The context was the industry standard. So the most efficient, IS the way to do this. No business is going to thank you for using their time to learn something. Not to mention deliberate obscurification, for the next person who may have to update it. Other people (Evil Phil) have suggested exactly what I did, I was just agreeing. So there is no reason to update a 6 month old thread to quote me. With respect. Thats bad webetiquette.

unknownlegend

I'd use the entity framework, c#/sql2008r2. There's no point rewriting the wheel, and in .net 4 it has received significant performance and efficiency upgrades compared to .net 3.5.
Linq makes it easy to retrieve, manipulate etc.

That's my recommendation. however look into serialization if you're looking at storing objects for future retrieval.

BostonB

The problem comes when the student who has designed such as system is off in Australia, and you're shop or business, has a problem when the system goes down. Who do you get to fix it. Will the owner be able to fix it themselves.

Dave!

Related question that ye might be able to help with

I'm working on something at the moment that uses an autocomplete (jQuery plugin) for a 'city' search box. I was going to store the list of cities in an XML file, and just have the plugin use that each time it's called, but for some reason it seems to have to use a PHP script, or else an array stored on the page that the autocomplete is used on.

At the moment I'm having the PHP script grab the list of cities from an XML file, and echo them for the JS to retrieve, but it takes ages if you use any kind of big list! Would I be better off storing the cities in a DB, and retrieving them from there?

Cheers!

BostonB

Dave - I can only tell you what we do. If it needs to be fast and the data is relatively static then it goes into the local lookup. If the dataset is large, or dynamic we use the database. If thats slow then we look at the bottle neck, can we rewrite the query, use a view, stored procedures, indexes etc. There isn't a formal approach mores the pity.

Sparks

Mod note:

One of the first rules on boards is "Don't be a dick".
Don't break that rule in here please.
Posts deleted and posters infracted/banned for doing so above.