Would you call this song jazz or blues?

I dont generally get too hung up on labels, but to answer your question :

Structurally it does not fall into either category IMO, though the tone of the guitar lends a jazzy feel to the song. I can also hear a kind of latin american (sun, sea, and sand ) flavour in there also.

What ever label you want to put on it, I think it's a nice song.