C# Reading from a file while another program is trying to get exclusive access.

BrokenArrows

I have a FileSystemWater monitoring a directory for changes.
When the log file is created/modified I am reading this file using:

FileStream fs = new FileStream(FullPath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)

I have a problem with the program which is creating/modifying the log, it is opening the file using the FileShare.None parameter of a FileStream. This means that if it attempts to write while im reading the file it will throw an IOException that the file is in use by another process.

Unfortunately i cannot modify the 3rd party application to use FileShare.ReadWrite so im trying to figure out a way to read the file without breaking the 3rd party log writer.

Is it possible?

John_Mc

Could you copy the file into a temp location and read from that instead? Not the most elegant solution but your hands are tied given that you can't change how the file is written to.

BrokenArrows

Copying the file still requires reading of the file and causes the same problem and i expect the file to get quite large so copying will take longer each time.

Colonel Panic

This highlights one of my problems with how C# uses exceptions for things that aren't exceptional and forces you into using them to control program flow.

Basically, you need to eat the exceptions in a while loop (add a timeout value too) until you can get read access, then do your thing.

You still suck at Googling btw

http://stackoverflow.com/questions/10982104/wait-until-file-is-completely-written

dazberry

Have a look at Volume Shadow Copies from .NET. Might be something there to allow you to get a copy of the file while locked.

D.

BrokenArrows

Colonel Panic wrote: »

This highlights one of my problems with how C# uses exceptions for things that aren't exceptional and forces you into using them to control program flow.

Basically, you need to eat the exceptions in a while loop (add a timeout value too) until you can get read access, then do your thing.

You still suck at Googling btw

http://stackoverflow.com/questions/10982104/wait-until-file-is-completely-written

You still suck at reading the OP!!
The problem isn't with me reading the log, its with the other program writing to the log while im reading it.

Colonel Panic

LOL! Touché

BrokenArrows

dazberry wrote: »

Have a look at Volume Shadow Copies from .NET. Might be something there to allow you to get a copy of the file while locked.

D.

Ill have a look at this tonight and see if I can get it working. Thanks.

tobsey

Can you post the name of the third party library? A logger that exclusively locks a text file sounds pretty bad. Are there alternative configurations so that it can log to a different storage format, such as a DB or something?

Is it possible to suspend the application that the logger is called from? Not the cleanest but your handler for the watcher could perhaps stop the service, copy the file, restart the service and then process the copied file.

This would really depend on how active the logger is though.

tobsey

Colonel Panic wrote: »

This highlights one of my problems with how C# uses exceptions for things that aren't exceptional and forces you into using them to control program flow.

Basically, you need to eat the exceptions in a while loop (add a timeout value too) until you can get read access, then do your thing.

You still suck at Googling btw

http://stackoverflow.com/questions/10982104/wait-until-file-is-completely-written

C# is not throwing the exception. The exception is being thrown in unmanaged code, which is then caught the System.IO.FileStream class and wrapped in a new exception. The underlying Windows API for accessing the file system is the problem.

McDonnellDean

It would make more sense to log to memory first and then to a file at regular intervals no? That way you should be reading from memory first. The logger I am working on logs in a similar fashion to how an event aggregator works. It would be up to individual handlers to log to a file or not. Logging to memory kills the need to read files and the only exporting done is via a handler, that way the handler can decide how to do file access, be it batched or per line.

I am putting the finishing touches on it and will have it on Github in a week or two. I'll post the link here when done.

amen

Logging to memory kills the need to read files

and what happens when you lose the in memory entries ?

or you have users who need to view the logs but don't have the programming knowledge to access them via a handler ? What about production systems?

McDonnellDean

amen wrote: »

and what happens when you lose the in memory entries ?

or you have users who need to view the logs but don't have the programming knowledge to access them via a handler ? What about production systems?

In a production system you would have multiple handlers, once such handler would feed a log view which would break the logs into filterable searchable entries for that view. This is what we do.

In terms of memory loss, the write handler is designed to be invoked on crash, unhandled exception included. We don't use files as persistence so power cuts wont effect us. Writing to a file is actually just an export to file handler, it cues up log entries, prunes them to look nice in text format and then batches them all to a file at once.

Interestingly we don't use levels or verbosity anymore, instead we use tags, similar to music, these are alphanumeric and case insensitive, it has vastly improved our facility to log.

McDonnellDean

Of course, re reading the question, it will still not solve the issue the user has which is reading in from a third party.

amen

Interestingly we don't use levels or verbosity anymore, instead we use tags, similar to music, these are alphanumeric and case insensitive, it has vastly improved our facility to log

thats an interesting approach. I might consider the use of Tags.

McDonnellDean

amen wrote: »

thats an interesting approach. I might consider the use of Tags.

We added it as an after thought to allow people to group related functionality together because we needed to filter the logs somewhat to keep them readable. It turns out that it is the backbone of our logging now and made the viewing of logs by end users really powerful.

Essentially users can filter a grid by tags and get a very narrow view over a huge log sample, which they are delighted with.