Optimum Structure for File System?

MeatProduct

I'm upgrading my computer soon to a dual dual-core opteron system. With all that speed I'm wondering if there is an advantage in having the following kind of setup:

1 hard disk for the windows OS
1 hard disk for all my applications
1 hard disk for storage and scratch/swap space.

Would any of those see a worthwhile benefit from a RAID 0 setup?

My work is in 3D rendering/modeling and a lot of Photoshop usage.

Many thanks,

Nick

WexCan

You could stripe all three disks and set up a main partition for the OS, say 10-15gig, then partitions for apps and storage.

Or have 1 disk for OS and stripe two, or have three seperate disks.

The possibilities are endless, maybe someone using their sys for similar purposes might share their config?

commited

I'd put 2 in raid definately - for the OS and apps - storage doesnt need to be crazy fast. If you can, get a small quick hard drive for your pagefile

MeatProduct

WexCan wrote:

You could stripe all three disks and set up a main partition for the OS, say 10-15gig, then partitions for apps and storage.

Or have 1 disk for OS and stripe two, or have three seperate disks.

The possibilities are endless, maybe someone using their sys for similar purposes might share their config?

Thanks WexCan,

What would be the advantage of partitioning in the case of striping all 3 drives? I thought it would improve the overall speed of the system if the various areas (os, apps) were on separate channels? I've no problems getting a load of extra drives for this but would that just be a waste of money (so I could have stripes for each file area)...

Thank you,
Nick

MeatProduct

commited wrote:

I'd put 2 in raid definately - for the OS and apps - storage doesnt need to be crazy fast. If you can, get a small quick hard drive for your pagefile

Are you suggesting this?

2 drives in RAID 0 for the OS
2 drives in RAID 0 for apps (or just the 1 drive or shared on the OS drives?)
Whatever for storage
1 fast drive for swap/scratch

Thank you Commited,

Nick

WexCan

MeatProduct wrote:

What would be the advantage of partitioning in the case of striping all 3 drives?

Just having seperate drive letters to keep things tidy, really.

commited wrote:

I'd put 2 in raid definately - for the OS and apps - storage doesnt need to be crazy fast. If you can, get a small quick hard drive for your pagefile

Sounds good - so 2 disks in Raid 0 shared for OS and apps, whatever storage you need and a quick l'il thing for swap.

Mutant_Fruit

But striping 3 drives in Raid 0 makes it a hellofalot more likely for you to lose all your data. If one drive dies, everything is gone!

StRiKeR

I'm running Raid0 C: system and apps, raid0 storage and Raid0 E: temps and swap file and current projects I'm working on, seems to perform much better then the old single drive setup

SouperComputer

Your swap file should be as close to the start of the drive as possible as it is the fastest part of the drive.

DemonOfTheFall

Definitely don't keep your work on RAID 0. Or if you do, do daily backups to a more secure media. RAID 0 is asking for data loss.

SwampThing

It's always going to be a trade-off between performance (RAID 0) and resilience (RAID 5).

The thing to remember is that speed comes from writing/reading data over many different physical drives. If you're going to use more that one disk for a logical drive, personally, I'd try some sort of hardware RAID solution instead of striping any day.
If hard drives aren't an issue for you, then you'll probably be able to afford some class of a RAID controller.

You haven't said what size drives you have.

For a starting point, what you said yourself is fairly valid. For OS drive, two disks in RAID 0 gives you good performance. Your accepting that if one drive fails, you're prepared to re-install the OS. If you have to re-install the OS, you'll have to re-install the apps as well, so there's little point in creating a separate RAID 0 volume for apps/games.

For your data however, you shoud look at RAID 5 or RAID 1. This is a home PC, so maybe a pair of large capacity disks in a RAID 1 set. Granted, you 'lose' one disk, but resilience is the order of the day. If you've the moolah and the space, then use 3 identical disks in RAID 5 - again you 'lose' one, but are safe in the event of a disk failure.

A good RAID controller is key to this working or not. Make sure it can support whatever RAID config(s) you want and the number of physical disks you want to connect to it. You get what you pay for with these devices, so don't be afraid to splash out a bit on a good controller.

As well, don't get too hung up on putting the page file on fast storage. Having it on separate storage than the OS is a good thing - so is lots of RAM

... forgot to ask - are you using IDE or SATA or SCSI? What's you're current config?

SwampThing

SouperComputer wrote:

Your swap file should be as close to the start of the drive as possible as it is the fastest part of the drive.

Souper, are you still using that Wang SX 16Mhz machine, with 4MB RAM and the 40MB drive?

SouperComputer

SwampThing wrote:

Souper, are you still using that Wang SX 16Mhz machine, with 4MB RAM and the 40MB drive?

Never had a Wang TBH, I guess I wasn't cool enough!

Anyway, I stand by what I said, if you are using a swap file it should be at the start of the drive as it is the fastest part. This is reflected on hardware level tests of drives, some have as much as 20MB\s difference from start to end.

Ideally the swop\scratch should have its own drive\array but fialing that and the budget to buy a crap load of ram, keep the swap file on the first partition.

CyberGhost

1 Drive: C:\ + apps
2 Drive: Storage

Is it worth partitioning C drive to allocate a separate partition for swap file? is speed increase that noticable?

MeatProduct

Thank you all indeed for your feedback.

My current setup is the following:

2 x 160GB ATA drives in RAID0 partitioned into "Apps" and "Storage"
1 x 80GB SATA drive partitioned into "OS" and "Development"
1 x 80GB SATA partitioned as "Media", kind of pointless I know but couldn't get windows to boot of the SATA RAID config.

I'm taking your advice SwampThing about getting a separate controller. I'm looking to go with Raptor drives but I haven't shut the door on SCSI but I'm not entirely sure my work would benefit from it.... I had thought having Raptors in a RAID0 setup would be plenty fast for my needs. My development work can get rather large in size (just finished rendering to an uncompressed avi and it takes up 3.2GB, took 72 hours to render also, hence the upgrade) but I can just backup old files as it's very rare that I would need them again. 36GB drives would probably do me well.

So RAID5 for my "Development" stuff then, I'll go with that, will have to get a controller in that case.

SouperComputer, I might go and use a 36GB Raptor just for swap/scratch space so nothing else will be using that drive. I wasn't aware that it's important for the data to be at the start of the drive, thanks for that.

I'm getting 8GB of RAM, Photoshop can really eat it up. I'll need to be getting Windows 2003 Server 64-bit to address all that properly.

Thank you all indeed for your advice, it's rally helped me clear things up.

Nick

SwampThing

Meat, do your homework on the controller and the drives, especially the controller - I can't stress that enough.

SATA II is around now, with very high throughput performance. It's most definitely worth looking at - not sure if there are motherboard implications with it though. Even so, SATA is very much a viable option. With the right controller, you'll get comparable performance to SCSI for a fraction of the price.

If massive capacity is not an issue, then don't waste money on huge drives. Smaller drives, with big caches, tend to perform better. 72GB Raptors should, most definitely, be in your equations.

With your current drives - you could RAID 0 the two 80GB for OS and RAID 5 the 160's for data - just to start off.

And just to follow up on Souper's point about swap - he's absolutely right about the performance difference between inside and outside of a single disk. In saying that, if you've two 10K Raptors for you're OS drive, you're not going to see much/any performance increase in having the swap reside elsewhere - it doesn't much faster than that for desktop PC's.

Good luck with it.

Duffman

CyberGhost wrote:

Is it worth partitioning C drive to allocate a separate partition for swap file? is speed increase that noticable?

Putting the swap file on a seperate partition on the same drive can actually harm performance.

As SouperComputer says, having it at the start of the drive is the most important thing. Some defrag software like Disk Keeper can move the pagefile to the quickest part of the drive.

SwampThing

No argument there, if we're talking about a single drive. Chances are, if there are multiple partitions on a single drive and you put the swap file on a separate partition, it'll be nowhere near the beginning of the disk.

I'd hazard a guess that 99% of drives configured like that have the C drive as the primary partition; as the first partiton on the disk; and any other 'drives' in the extended partition.

I've never actually tried to mark any other partition, other than the primary, as Active. Does it work?

I think the location of the swap file should only be of concern if you're looking at a machine with multiple drives, and even then, only if the configuration allows you control the exact location on the physical disk that the swap file will reside. For example, on a RAID 5 volume, who can say where any data will sit on the physical disk?

commited

MeatProduct wrote:

Are you suggesting this?

2 drives in RAID 0 for the OS
2 drives in RAID 0 for apps (or just the 1 drive or shared on the OS drives?)
Whatever for storage
1 fast drive for swap/scratch

Thank you Commited,

Nick

Well if you have the cash, thats definaltely a fantastic idea, but i'd be more inclined to have 2 fast drives in raid for my os & aps, like raptor, (partitioned half and half or whatever) with 1 small drive for page file & then a big hard drive or 2 for storage

shayser

Apologies for butting in here... quick question... I've a similiar decision to make. Plan is for 3 x 72Gb 15000rpm SCSI RAID5 "working" drive + 1 x 300Gb SATA for storage. How would that go? With Dual Core + 4Gb memory would a scratch drive add a whole lot more performace?

SouperComputer

2 drives in RAID 0 for the OS
2 drives in RAID 0 for apps (or just the 1 drive or shared on the OS drives?)
Whatever for storage
1 fast drive for swap/scratch

One RAID 0 of 76GB is sufficient for both OS and Apps.

Whats more important, from what I can gather talking to folks working with poster-size images is that your scratch is on a seperate physical drive.

Have you picked out a RAID controller? Bear in mind that most SATA onboard controllers are softraid, not fully hardware and should be avoided if you can.

How you throw your RAID arrays around really depends on your everyday use of the computer. You may find that RAID for the OS suits you best with better application launch times. or you may find that you are dealing with huge images and a RAID scratch provides better productivity.

I think start off with two non-RAIDed Raptors or SCSI Atlas 15k II drives, the performace may be sufficent for you:

1. OS\apps
2. Scratch\swap

Then have a storage drive if needed, of course if the info is of importance then maybe consider a mirrored 1 array for this.

Capt'n Midnight

If you are using small images , depending on how much RAM you have it might be better to look at it first as it will be 100's of times faster than HDD's.

Look at the max memory used. If using Windows then use taskmgr/performance to see the maximum memory allocated after a heavy session - if commit charge is less than the amount of RAM you machine can take than I'd say more RAM first to take the load off the drives.

or an i-RAM card - not sure where you'd buy one -http://www.theregister.co.uk/2005/10/03/gigabyte_i-ram_ramdisk
but should be good for swap / scratch - not because of the data transfer rate but more for the low access time in that you don't have to wait for the drive to spin around again.

MrPinK

SwampThing wrote:

If you have to re-install the OS, you'll have to re-install the apps as well, so there's little point in creating a separate RAID 0 volume for apps/games.

I still always like to seperate them. I usually have a 5 gig partition just for XP and nothing else. This lets me reintall XP quickly at any time without having to worry about backups. Yes, I'll have to run the installations again for all my apps/games, but the configs, settings files, and save games will all be exactly as I left them. No need to back them up beforehand and then restore them afterwards. A quick and easy new system build.

Balfa

SwampThing wrote:

If you have to re-install the OS, you'll have to re-install the apps as well, so there's little point in creating a separate RAID 0 volume for apps/games.

Very few apps I use need to be reinstalled after an OS reinstall. And I think it makes things much neater to have them on separate drives.

MeatProduct

SouperComputer, I'm looking at getting this controller:http://www.komplett.ie/k/ki.asp?sku=314640&cks=PRL

Capt'n Midnight, that I-RAM card looks very good. Might go for one of them.

Thanks,

Nick

SwampThing

Meat, are you sure your motherboard take PCI Express cards? Will it support SATA II and will the SATA II controller be happy with SATA I disks?

All questions someone else will be better able to help you with, but they need answering before you buy.

MeatProduct

SwampThing wrote:

Meat, are you sure your motherboard take PCI Express cards? Will it support SATA II and will the SATA II controller be happy with SATA I disks?

All questions someone else will be better able to help you with, but they need answering before you buy.

Hi SwampThing,

Thanks for that, I'm quite happy with my motherboard choice, it's the Tyan Thinder K8WE without the onboard SCSI. It has all the ports and slots I'll need. Here's a link to it:
http://www.gamepc.com/shop/product.asp?catalog%5Fname=GamePC+Online+Products&category%5Fname=Motherboard&product%5Fid=TYN%2DS2895

Thanks,

Nick

SwampThing

Cracker mobo Meat. That's gonna be one hell of a rig.

MeatProduct

SwampThing wrote:

Cracker mobo Meat. That's gonna be one hell of a rig.

'Tis going to be a cracker of a price too!