research tool

AidanD12

A friend and I (both data scientists) are starting a project to develop a software tool to assist the small investor when researching stocks. The intention is to do sentiment analysis (via machine learning) using the live input of multiple social networking and discussion groups and combine with any relevant news from media feeds and also the normal financial assessment figures (such as p/e etc) to come up with a stock rating/recommendation. The current stock holdings would be known along with a client profile (such as risk aversion etc) so that we could recommend a stock to keep a balanced portfolio.
We’d really appreciate some feedback from the investor community to see if this is something that they would be interested in or see as potentially adding some value.

ixus

First recommendation is to research your competition.

Sentiment indicators are not new. I'd say well over 10yrs.

What you describe doesn't bring anything new to the market.

Next, if it's not new, it has to be better. First, you'll have to backtest to some degree. Then you'll have to demonstrate in a live market for a period of time to show it's not just curve fitting.

All of this will require content on a daily basis, building up a following which ultimately converts to paying customers.

Then you just have to maintain a positive track record.

ðŸ˜

Jim2007

AidanD12 wrote: »

A friend and I (both data scientists) are starting a project to develop a software tool to assist the small investor when researching stocks. The intention is to do sentiment analysis (via machine learning) using the live input of multiple social networking and discussion groups and combine with any relevant news from media feeds and also the normal financial assessment figures (such as p/e etc) to come up with a stock rating/recommendation. The current stock holdings would be known along with a client profile (such as risk aversion etc) so that we could recommend a stock to keep a balanced portfolio.
We’d really appreciate some feedback from the investor community to see if this is something that they would be interested in or see as potentially adding some value.

This kind of think is ten a penny and so far has been a dismal failure and is likely to remain so. Social media is properly the biggest enemy of the small investor, it is actively used to talk up or down stocks by the sell side. Perhaps you might be able to sell the idea to some of the sell side organisations as it would add to they tools nicely.

Kickstart1.3

Sounds interesting,
lots of tools out there, I did a document parser for an investment firm years ago while I was in college. Basically they wanted to screen RNS press releases and gauge the tone of the release. They were really happy with the results of what I done, and no I didn't get paid a fortune from them it was quite modest as I only worked on it for over a week. That was nearly 20 years ago
Since then I do know they have gone on to do more advanced systems and have a whole team of computer scientists working on software for them. I can only imagine how advanced the software has become, but at a guess they would have AI reading and participating in forms and chat rooms.

AidanD12

Thanks very much to you all that took the time to reply. Any other inputs greatly appreciated. We're at a concept stage and trying to assess the market.

ixus wrote: »

First recommendation is to research your competition.

Sentiment indicators are not new. I'd say well over 10yrs.

What you describe doesn't bring anything new to the market.

Next, if it's not new, it has to be better. First, you'll have to backtest to some degree. Then you'll have to demonstrate in a live market for a period of time to show it's not just curve fitting.

All of this will require content on a daily basis, building up a following which ultimately converts to paying customers.

Then you just have to maintain a positive track record.

ðŸ˜

Agreed on pretty much all counts ixus. The idea of sentiment analysis has been around a considerable time but has not been used successfully so far as I can tell. There has been a number of interesting research papers in the last 12 months which may change this. The issue with sentiment is that it's instantaneous. Previous models tried to measure sentiment today and predict for tomorrow .. which is too late as the market has already moved. With live streams of data and the processing power/bandwidth to handle it along with the appropriate algorithms to decypher it, the results should be better ... but of course this will have to be proven.

The competition is quiet fragmented and hard to assess. There appears to be a lot of work being done by hedge funds but we have no clear visibility on this. There are a few startups dotted around the globe working on it bit no actual product that I can find yet

Jim2007 wrote: »

This kind of think is ten a penny and so far has been a dismal failure and is likely to remain so. Social media is properly the biggest enemy of the small investor, it is actively used to talk up or down stocks by the sell side. Perhaps you might be able to sell the idea to some of the sell side organisations as it would add to they tools nicely.

Yes lots of painful sides to social media like you describe Jim. I guess people always have been trying to influence and manipulate the market by whatever means they can. Our idea is to try and ignore the spam and give an edge by picking out the 5% important information from the rubbish. Like you say more failures than successes. James Dyson always a good guy to quote on failures .. think he had 5000 before he eventually struck gold!

Kickstart1.3 wrote: »

Sounds interesting,
lots of tools out there, I did a document parser for an investment firm years ago while I was in college. Basically they wanted to screen RNS press releases and gauge the tone of the release. They were really happy with the results of what I done, and no I didn't get paid a fortune from them it was quite modest as I only worked on it for over a week. That was nearly 20 years ago
Since then I do know they have gone on to do more advanced systems and have a whole team of computer scientists working on software for them. I can only imagine how advanced the software has become, but at a guess they would have AI reading and participating in forms and chat rooms.

Thanks Kickstart. I've just finished a masters in AI and it's amazing how quickly the landscape is changing in the technology. Lot's of scary stuff like chatbots and the likes that make the headlines but lots more very positive stuff

Jim2007

AidanD12 wrote: »

Thanks Kickstart. I've just finished a masters in AI and it's amazing how quickly the landscape is changing in the technology. Lot's of scary stuff like chatbots and the likes that make the headlines but lots more very positive stuff

I started working on AI projects in the financial field over 25 years ago and yes the landscape has changed dramatically over the period, but the scary thing is that the outcomes are no better today than they were then!

We still have no accurate way to model an experts intuition and the mathematics of finance creates a sense of confidence where none should exist.