I think there are some really interesting analyses to be done here.

Social media often use novel abbreviations and alternate spellings, so lexicons need to allow for that. That's pretty easy, though.

The trickier part is determining how social media text differs syntactically. I'd bet that some but not all function words are often left out; but perhaps that is only in certain contexts. "..." is very common, but just what can it substitute for? The whole sense of a text can be drastically change by one emoticon added to the end. And at least in some social media, a text may be divided across successive messages, with no machine-readable indication that the parts go together.

I don't think it's well understood yet, what the similarities and differences are between various kinds of social media. There's room for a lot of very interesting study here.

Steve

Another challenge lies in the simple fact that short posts, notably tweets, often have sentiment, etc., but no topical content. Example: "That's great! I love it." Given this, we are looking into producing document-level polarity scores, so that at least the overall polarity of the text can be discerned. Look for this in a future release.

I would be willing to bet that social media, at least as much if not often more than other traditional media, is laced with humor and sarcasm.  Also, conversations are not as often threaded which makes for a huge problem with co-reference.

I'm not a doctor, nor do play one on TV, but I do have some history writing medical record-keeping software. Some of the hurdles we encountered  (not insurmountable, but definitely there):

Almost every medical institution (hospital, practice, clinic, etc) uses different standards for their records; what needs to be recorded, what things are taken for granted, details of terminology, etc. In my prior job, we dealt with this issue by making our system extremely customizable. In terms of OpenAmplify, I think that would mean being able to vary some of our underlying ontology etc. according to what data stream we were parsing at the moment. The ongoing effort by the HL7 folks to standardize electronic patient data transmission might help us some too.

A lot of doctors are very resistant to doing anything with or on a computer; they tend to view it as clerical work which is beneath them. This has been changing slowly as more post-internet era medical students turn into doctors, but it's going to be while before the problem goes away completely. In the case of the sort of thing you're talking about this may not be a big issue, since we'd be analyzing existing reports, not asking people to create new ones.

This approach could be used on news sites -- most of whch have links back to prior articles on the same subject (usually created by hand, I'd bet), but none that go forward. Or, it might be useful to filter out the 200 slightly-rewritten versions of the original NYT story that inevitably show up when you search.

Interesting! I have also been thinking a bit about this lately, and I think that especially Actions can be really useful for this. After all, the "summary" of an article, at least news content, is often expressed using an action:

The president is giving a speech in Egypt.

If we find reoccurring actions, we should often find articles describing the same event.

Now  "Can we think the other way?" Find all article/pages/websites/ that talk about  similar topics. One obvious use would be in Twitter . ( courtesy @RamS)

Yup !!! , We can identify people with similar interest of yours and you can decide whether  to follow them or not. Rather than wasting time in following and then unfollowing ,we can have an application which takes  the tweets of the user , amps it ,compares with yours( yours tweets are also amped and you do have an amplify output). 

We built a initial prototype and the results were decent. The only problem was amplifying the tweets . We were amplifying the latest 25 tweets and analyzing it with the person you would like to follow. It also holds good at times , as it tells your Current interest (say it in a Techie/Funky way - Current Trend) .

A  FF Addon for this would be awesome, which takes a username and gives  you

1. The Interest score 
2. A button which has Follow/UnFollow label based on 1

The above answers Mr.Tweets Problem as well
http://blog.mrtweet.net/follow-all-you-want-but-would-you-follow-yourself

Is there any place I can go to see how you are comparing items for similarity? I am developing an application that I am integrating with OpenAmplify to determine document similarity, above and beyond the phrase matching that I already use.

Thanks!

J.Ja

"An easy way to start might be to calculate the difference in weight assigned to each topic found in both, and discount for the weights of topics that aren't found in both. Likewise for actions and intentions."

I tried this approach this weekend with the Demographics and Style sections, with good results. For my purposes, I did not need to be terribly exact, I just ensured that the "Name" on each sub-element was the same.

On my sample documents, I am going to roll up my sleeves and compare the topics, actions, and intentions, and see how I feel about comparing the numbers. For my needs, I am actually being pretty darned specific in what I am looking for... I am looking to see if the two items could be extremely similar in content without necessarily being similar in wording. So if they cover the same topics with roughly the same intentions and the same topics, I feel pretty confident that they may be rewordings of the same document.

J.Ja

Do you mean in terms of common topic, polarity, etc.? This is a great idea--document similarity is huge for Google, Yahoo, etc. of course, for search results. The usual approach is lexical similarity (common words or many words in a common semantic neighborhood). But effectively OpenAmp[lify does the same when it determines topic--and the addition of other metrics would be a bonus!

How does it calculate topic and intention similarity? I'm surprised by the zeros in Ram's example...

It should really be called "how many things from the first document appear in the second document" instead of similarity, because that's all it does :) And the topic intentions are buggy, I don't think it compares them all right now..have to take a look as soon as I find some time! The whole approach of this is really too simplistic, and it would be fun to take it a bit further and try clustering a bunch of documents using these features..

On the other end of the simplicity spectrum, here is a recent article on a document similarity measure. There are many different metrics used and some articles provide comparison statistics. 

http://web.jhu.edu/HLTCOE/Publications/acl08_elsayed_pairwise_sim.pdf

You should actually be seeing a series of articles coming out soon about this! In fact, I just figured out exactly how to do the comparison. Or, I should say, I just remembered something I read a long time ago, meant to do something with, and just not realized that it is 100% applicable to the task at hand.

For my purposes, I am looking for things which are copies of documents above and beyond revisions, but including a full rewording. That's where OpenAmplify really comes into play. If I find a document that only has a few phrases in common with my source document, but OpenAmplify shows that they are nearly identical in topics, guidance, etc., then I can then do things like try to do an in-depth thesaurus comparison.

J.Ja

Hi Justin,

I'm not sure I follow your reasoning on weighting by depth. The domain nodes, for example, provide more granular information near the children; a matching topic with matching polarity seems like a stronger indicator of similarity than just a matching top-domain? But on the other hand, you can't do much by just matching the polarity nodes, they only make sense in the context of their topic, so I think just matching nodes and scoring based on depth won't do you you much good.

My guess is that the key is to find the right algorithm to compare each signal structure on it's own; for topics you could compare a complete topic subtree and award one point if the actual topic matches and bonus points for matching polarity, guidance, etc. 

Bu maybe I didn't completely understand where you were going?

janeS:

On the other end of the simplicity spectrum, here is a recent article on a document similarity measure. There are many different metrics used and some articles provide comparison statistics. 

http://web.jhu.edu/HLTCOE/Publications/acl08_elsayed_pairwise_sim.pdf

Justin,

I think your basic approach of taking each node from the first document and then searching the second document for matches should work.  A couple of thoughts:

1) The comparison function for Document level scores (like Styles and Demographics) is different than the function for Feature level scores (like Topics, Actions and Topic Intentions).  For the Document level scores you want to see if the double in the matching <value> tag is within a certain "similarity threshold". 

2) For the feature level scores you need to match the string in the <name> tags, but because the score in the <value> tag is not normalized across documents you would need to select another heuristic to determine an additional weight to the co-occurrence of terms.

Adam -

Hmm. Part of the problem here is my poor memory! I last looked at OpenAmplify a number of months ago, and I am in the planning stages of my project, and I seem to recall there being a deep structure. I guess I was mistaken! I like your suggestion though, and I'll give it a whirl this weekend.

Thanks!

J.Ja

ramS -

Thanks! I will be sure to keep this in mind. I hope to do this implementation over the weekend.

J.Ja

I think we may have 2 threads going at the same time:

* One seems to be comparing two arbitrary documents -- sort of an XML diff -- I think that's what Justin's last post was referring to.

* The other is comparing the OpenAmplify analyses of two arbitrary documents. That's a much easier comparison, because you don't really have to think about the XML structure at all -- just compare some of the numbers it returns.

XML diff has a long history, and there are lots of known methods; Fabio Vitali published something on this recently. Comparing the OpenAmplify analyses has the advantage that you're actually comparing topics, goals, etc -- so it should have a decent chance of noticing similarity even if the whole piece has been re-written or re-phrased. I'd say that even a moderate similarity there is significant.

An easy way to start might be to calculate the difference in weight assigned to each topic found in both, and discount for the weights of topics that aren't found in both. Likewise for actions and intentions.

Just as a quick experiment I put a dead simple similarity tool here:

http://79.125.39.138:8888/

Nothing clever so far, just compares two URLs to see how many of the topics, actions and intentions the articles have in common.

Here's the output of Adam's similarity test on the Washington Post and New York Times articles above.

Topic similarity: 44.4%

• president
• Bernanke
• crisis
• Obama

Action similarity: 0%

Topic intention similarity: 0%

sderose:

How does it calculate topic and intention similarity? I'm surprised by the zeros in Ram's example...

Let us know how this turns out.  Sounds like a really interesting application of the OpenAmplify engine.  Tracking down copies and older versions of documents on a file/email/content server can result in huge reductions in storage and backup requirements.

mikepetit:

The XML structure returned by OpenAmplify should lend itself to using the service to determine how similar texts are. This could be used, just to name a couple of examples, in automated polling or in brand management.

Slightly related thread about using OpenAmpify to find duplicate news articles:

  http://community.openamplify.com/forums/p/305/460.aspx#460

I think actions (and topic intentions) could be an interesting variable in general document similarity as well.

Seems to me a basic module that traverses two XML input streams and returns a percentage match would be a great way to test this. Anybody want to try this? "Topical match: 69% Intentional match: 85%". Styles could, I believe, be safely ignored, as could demographics. This would actually permit us to compare the core message of the two texts without regard to differences in authors' styles and characteristics. The more I think about this, the more I like it. Just run it against TopicIntentions. I have a hunch it would prove very enlightening.

Volunteers?

I think my initial idea was something closer to Microsoft Single Instance Storage.

http://www.microsoft.com/downloads/details.aspx?FamilyID=99f8ee58-4faf-4951-ba84-7237b5c639b5&DisplayLang=en

I am trying to do something just like this in my application. My idea is to walk the tree of the source document, and for each node in it, find an identical node in the target document. In each pair of matching nodes, walk them and tally up the number of matching children, and provide a score for the found node. Then, add up each node's score in the document, weighted by depth so that nodes on near the root carry more weight than nodes amongst the children.

Thoughts?

J.Ja

Here is the OpenAmplify TopicIntentions analysis of the coverage of the President's announcement that he was nominating Ben Bernanke to a second term as Chairman of the Federal Reserve in the New York Times and the Washington Post.

New York Times

Main Topics and Intentions:	Attitude:	Intention:	Guidance:	Timescale:
1 Bernanke			Offering
<- communicate				Future
-> communicate				Past
<- assess			Offering	Recent Past
<- choose			Offering	Past
<- help				Past
<- want			Offering	Past
<- socialize				Past
2 Obama			Offering
-> choose			Offering	Past
-> vote				Future
-> communicate				Future
-> request			Offering	Past
3 president			Offering
<- believe				Past
-> believe			Offering	Present
-> vote				Future
4 U.S.
-> communicate				Present
<- compete				Present
5 economy			Offering
<- help			Offering	Present
<- socialize				Past

The Washington Post

Main Topics and Intentions:	Attitude:	Intention:	Guidance:	Timescale:
1 Bernanke			Offering
<- create				Past
-> create			Offering	Future
<- move				Past
-> move				Present
-> help				Future
<- pursuing				Present
<- request				Past
2 Obama			Offering
-> communicate				Present
<- move				Past
-> move				Past
<- request			Offering	Past
-> pursuing				Present
-> socialize				Present
-> vote				Past
<- admire				Present
<- say			Offering	Present
3 chairman			Offering
-> move			Offering	Present
4 Ben Bernanke
<- communicate				Present
<- want				Present
5 economy			Offering
<- help				Future
<- move				Past

Since I was amused by all this, I looked more into Groucho's quips. I wonder if he was an amateur computational linguist? He certainly understood the biggest problems we face, things like PP attachment, etc. Amazing.

I'd buy that shirt... one size Large, please! :)

Ha! I knew there was a reason I loved it...Groucho rules. I have agreed to hold back on making a shirt out of it until we can amp it more accurately...after all, a "time fly" is rather uncommon.

Mike :-)

I can't help but notice that Groucho Marx was a master of using syntactic ambiguity:

"One morning I shot an elephant in my pyjamas.
How he got into my pyjamas I'll never know."
-- Groucho Marx

Lucky for us that we are not resolving PP-attachment (yet!).

HA! That’s wonderful…perhaps we should consider that first sentence for the T-shirts…

Mike :-)

> I can't help but notice that Groucho Marx was a master of using
> syntactic ambiguity:

They all were; that's what makes Groucho/Chico conversations marvelous.

Also impressive was Harpo's ability to do puns without speaking.

-John

> Ha! I knew there was a reason I loved it...Groucho rules. I have
> agreed to hold back on making a shirt out of it until we can amp it
> more accurately...after all, a "time fly" is rather uncommon.

How exactly would you do this? Try to incorporate a rule that "time" 
as an adjective is rarer than "time" as a noun? Then you'd get "time card" etc. wrong. Maybe "time" can't modify animate things? Then mark all words in the lexicon as "+/-animate"... and hope that "time" never actually does modify anything animate. And this rule would take care of only one word, time...

Nancy

PS By the way, there is a third parse of the famous sentence: "Herd flies like mosquitos". ("Herd" as a verb with implicit "you" as
subject.)

How about anagrams of OpenAmplify:

A few selected ones….

Famine Polyp
Amplify Nope
Amplify Peon
Foamy Nipple
Leaf Imp Pony
Flea Imp Pony
Piny Flea Mop
Leafy Pin Mop
Pony Lip Fame
Mealy Fin Pop
Ape Film Pony
Flay Pine Mop
Flay One Pimp
Flay Open Imp
Flay Imp Peon
Foam Pine Ply
Foamy Pen Lip
Manly Pie Fop
Pal Fine Mopy
Play Fine Mop
Lay Fine Pomp
Many File Pop
Many Life Pop
Many Pie Flop
Mayo Pen Flip
Map File Pony
Map Pony Life
Map Fine Ploy
Amp File Pony
Amp Life Pony
Fine Ploy Amp
Yelp Amp Info
May Open Flip
Any File Pomp
Any Life Pomp
Any Poem Flip
Any Pope Film
Pay Elfin Mop
Pay Felon Imp
Open Film Pay
Pay Peon Film

Short texts are much harder than longer ones in some ways, since sometimes you don't have enough information to determine things like topic or the correct sense of a word or term. At the same time they can be a lot easier because there is only one point to the text--so, for example, it should be pretty straightforward to see if someone is asking for guidance in a 1 or 2 sentence message.

I find the issue of text size to be especially intriguing these days. The advent of micromessaging in such forms as Twitter forces us to try to discover meaning in very small packages. Yet, as a human being, I can still discern a lot from 140 characters. In a way, our auccess in closing the gap between human and machine understanding could have no better benchmark than short texts.

Short texts are definitely a hot topic. They differ in so many ways: spelling, punctuation, word choice, syntax, how pronouns are used,...

I wonder how rapidly the characteristics of Tweets change; is the syntax a fairly predictable "adjustment" from normal syntax, or is it being invented continually in new and varying ways (vocabulary surely changes that way, but syntax? Maybe).

Regular English only has about 1.3 bits per letter of "real" information (that why, for example, you can compress general text down to about that size, and no further). I'd bet the information rate is far higher for Tweets, but I haven't heard of anyone checking. Come to think of it, I'll go check now......

Alas, I wonder if the lack of immediacy implied by replacing direct statements with citations might not hinder adoption of such an application. Despite the lack of economy inherent in repetition, it requires less effort for both the author and the reader. And, of course, the use of citations prevents people from posing as authorities themselves, which certain types of people will consider disadvantageous ;-)

I guess it would be interesting to consider what the user profile would be for such an app...

I see  your point and I would agree entirely, if not convinced that good and bad comes from context.

UX is crucial, it's all about how something is presented. It is true that I would find myself crippled, only able to cite others in order to express myself. But that would not be the case. I can still express myself uniquely and in those cases the application finds that my intentions are not unique, ie. someone else already had this idea, I get suggested to cite or reference those other persons.

This would be done in such a way that I find my article enforced by references, citations and even statistics in case I am one of thousands who have the same thought.

Ie. writing something about a politician P that should do X, I could use:

1. a statistic saying that 30% of social flows believe X is good for environment.

2. a citation from P saying that environment will be top priority after health care.

So what just happened?

a. I saved myself a lot of writing in a convincing way, since these are "hard facts".

b. I get "assisted empirical argumentation" and can focus more on emotional penetration.

c. I just clustered/associated 30% of the social flows about X with one or more articles of P. This is net reduction of entropy(adding structure to data) by adding new data.

The UX should be that:

1. I judge for myself what I need to express in my own words.

2. I continuosly get assistance as I write my text. The application saves time and enforces my text empirically.

3. Eventually I will be rewarded (here's the prestige and authority) by being acknowledged to be a person that has an overall view of the "world" and seeing the relations between events, opinions and the overall "world order". I.e. a manager lets other people do the hard work, finding out things, then he/she uses those findings to draw conclusions, make decisions and insights.

The last point (3) would become more obvious when the added structure comes to use. My name would be rated higher the more useful structure(insights) I have created.

And lastly... I tend to write quite lengthy messages. With this message, as all other messages I write, I have already tried to reduce the length a lot. I could for instance write much more to explain exactly what I mean. But to keep it "short", I try to write what I think will make the reader extrapolate to what is unsaid. However this often fails :). Also, I have not said anything about how to technically achieve this application/concept. With such an application that I describe here, I would have been able to tell much more, since someone else has probably said it before me and also because I have previously written vaguely about this elsewhere.

HA! Here's what happened to threewords.me, just a little while ago: http://mashable.com/2011/01/19/threewords-me-acquired-by-domain-name-czar/

Hmm...so, tell me: what *is* the next, big social media news/search/hybrid/whatever we need out there? There's so much value inherent in all this social media content, but harvesting the value still seems to be held up by the sheer volume of it all, the poor signal to noise ratio, etc. What's the app we all need? Let's figure it out, and build it ;-)

I've been thinking about an application concept for about 2 years by now. It addresses the signal to noise ratio that you mention.

I believe that we need a conept of applications that reuses information rather than creating new information. The reuse of information shall also contribute to a reduction of entropy. So, when you want to express something you should express it mainly with help of citations rather than rewrite it.

Ok, that sounds like a retweet, right? But a retweet does not add anything. So with this concept you will be able to add short information in a way that clusters information.

The result is that you have for instance drawn a conclusion from 4 pieces of information by adding a few sentences which explain how they relate.

Correction: Retweet might add information but it does not contribute to entropy reduction..

Hi Arnold -

Good to hear from you!

Interesting company :-) What makes you believe they will have such an impact?

Although I can't ever comment on relationships, real or hypothetical, that are not publicly announced, I can't resist noting that there are thousands of API key holders out there. In most cases, we have no idea who they are or what they're doing with OA...so who knows?

Hi Mike!

Ok, I am not convinced they will succeed, but everyones' expectations seem to be skyhigh and "they've already suceeded", that's the reason I use the word "obvious" (slightly sarcastically).

However, considering the description of what wavii produces which would be, ie. an automated wikipedia/meme tracker/social news hybride... in my ears, that sounds like dawn of web3.0... given that it succeeds of course.

Personally, I think that what I have read about Wavii sounds too complicated. Complicated stuff has great usage depth but no one cares to use it. UX is crucial. On the contrary we have sites like threewordsme.com which has high UX but usage is shallow/pointless giving high impact and short life.

How good are the results?

Resulst are pretty OK. They don't have  a Neutral Signal and hence they add up to Positive.For example , I saw one of  the tweets

" MJ vs. Madonna? oh that is too hard. mj ftw?? maybe "

TweetFeel show it as Positive Signal whereas Tweetsentiments (which uses OpenAmplify)  shows MJ and Madonna as Neutral.

You can check the following URL for results

http://tweetsentiments.com/analyze?q=MJ+vs.+Madonna%3F+oh+that+is+too+ha...

Their results are so good that either they have a breakthrough technology or it's done manually:-)  The volume of tweets they process seemed to be very low and mostly outdated - not as realtime as they claim to be.

I appreciate that in the previous comment TipTop is mentioned as a new semantic real-time search engine.  We launched the beta version of this service recently at http://FeelTipTop.com  While we are far from perfect in terms of both the functionality and the interface, I would strongly encourage this community to take a look at this engine closely.  Our approach towards solving this problem is entirely different from anything that has been tried so far.  You can look at write-ups about us on various sites including our blog and at http://www.altsearchengines.com/2009/08/31/search-engine-tiptop-announces-new-features/  I am happy to chat with any of you, if you are interested in finding out more.  Thanks.

I'd like to pick up the discussion around something that came up in this thread earlier:

"RIP,
R.I.P, or 'rest in peace': How should that be analyzed? How would a
human analyze it? It is of course a negative thing that a person has
died, but rest in peace is a nice thing you say.

So, a simple short tweet like: "RIP Patrick Swayze", should that be analyzed as expressing positive or negative sentiment?
What do you think?"

Twitter is absolutely flooded again today of RIP messages, this time dedicated to the young actress Brittany Murphy. I am considering today - again - to add support for this kind of context in our Sentiment engine. However, opinions here in the office are quite diverse, regarding the flavor of this expression.. 

What would you say?

RIP Brittany Murphy = Polarity: negative or positive?

Personally I am inclined to give it a score for negativity.

RIP Brittany Murphy = Polarity: negative or positive?

I feel like that statement is positive, but that is just my opinion.

I took a moment to test this application this morning. I run a whole bunch of different queries, and just as Balaji says, they have no Neutrals. When I run the same queries through TweetSentiment, I get a lot of Neutrals, so we know they exist.

I am thinking, maybe they just output the stuff they feel secure about: Could it be that the only tweets that are presented are the ones that are definite matches of rules or wordlists for positive/negative? Not that they are right every time: The tweets below come out negative:

"I am organizing for health care. http://bit.ly/kJ0Gk #OFA Read this to know what's up with Health care reform."

"What's going on with Health care reform? Read today's health news here! http://bit.ly/Vh8St"

TweetFeel was one of the companies mentioned in this morning's NY Times article on Sentiment Analysis.

http://www.nytimes.com/2009/08/24/technology/internet/24emotion.html

Of course I will keep an eye on TipTop!
And I wish you all the best and good luck!

Thanks for your messages, Alexandra.  Your points are all excellent.  I will review the links you sent.  It will be great if you'd continue to visit TipTop (http://FeelTipTop.com) from time to time to see how well we are doing.  More thoughts on how we can do even better will also be much appreciated.  Also, please feel free to reach out to me anytime by e-mail: shyam at tiptopbest.com

Hi Steve, that was the issue here:

Currently, when spelled out: Rest in peace Brittany Murphy, it is assigned a positive sentiment (polarity score), from the word "peace".

Missing in our resources were variants like RIP, R.I.P. and R.I.P, now added, with positive sentiment.

I can still see use for negative sentiment analysis in cases like these: if you get out on twitter and see RIP-messages all over, it's clear something tragic has happened. What do you think?

Nice to hear from you again, shyamkapur!

We have quite busy days ourselves, but I'll be happy to take a look at your new interface, as soon as I get the time. Thanks for reminding me, it's always nice to get some new views and ideas from other actors. Hope your interface is a success,

best, Alexandra

Few more to the list but little organized

1. RankSpeed -Search by sentiments.
2. http://feeltiptop.com

RankSpeed is a search tool that does a sentiment analysis on the
blogosphere / twittersphere to find the best websites, the most useful
web apps, the most secure web services, etc.

I searched for NewsPaper, and i got Forbes and CNBC as top Result

Feeltitop wasn't that great and the UI was horrible

I also haven't been seeing results from TweetFeel for my username or any terms I've been trying out.  I wonder if they are still working out the kinks.

It appears to work on trends.  So unless someone has used #weinberg81 recently, a search for your username will inevitably fail.  Try other trend topics like "Obama" or "bicycling."

shyamkapur:

This is an extremely hard problem and no existing solution comes anywhere close to perfection.  I would argue that even humans would not agree in many cases on what should be positive or negative.

Yesterday Steve blogged about a new publication: Detecting Sadness in 140 Characters: Sentiment Analysis and Mourning Michael Jackson on Twitter, and I quote:

"With 6 raters rating whether these tweets expressed sadness (all
contained the word 'sad'), they have a Fleiss kappa inter-rate
agreement of only 0.561. Pretty low for what would seem like a really
easy task."

Well... as stated before: sentiment analysis maybe isn't such an easy task after all.

Thanks, Alexandra.  It has been a busy couple of months at TipTop as we continue to innovate in a number of directions.  Please come and check out our new interface soon.  We have recently added a real-time, semantic, social shopping experience which you must also check out.  Please send me your feedback.  Thanks.

Pretty cool! Unfortunately, I'm not popular enough to garner any feedback on my own name... however, I CAN follow what people think about Silversun Pickups (an incredibly AWESOME band)! Thanks for sharing this, Balaji!

Hi shyamkapur, and thanks for your input!

I played around with some of these tools again today. It is quite clear that sentiment analysis on short texts like tweets is not the easiest task to take on.

One of the main issues that people talk about today on Twitter is obviously the tragic death of Patrick Swayze. I run searches of his name in our own analysis engine, OpenAmplify, and compared with results from TipTop above. I have to admit though, that I was not overly impressed. Maybe you could tell us some more about how your tool is unique.

Here's some of my observations from testing TipTop:
The latest set of hits that I got at one point were the following:

Positive:

  May Patrick Swayze be at peace and his family find comfort in the sweet memories they made together while he was here in this world. 

Negative:

  RIP Patrick Swayze. Dirty Dancing will always be one of my favorit movies and Swayze one of my first celebrity crushes.

Remaining messages: (According to the a definition in the article above: "Tweets deemed less useful are placed in Remaining messages, and may include polls, advertisements and standard, plain facts".)

BREAKING NEWS: Kanye West just interrupted Patrick Swayze's death to say that Michael Jackson's death was much better.

How are the three tweets different from each other? Why is one negative and one positive, and what makes Kanye West's crazy ideas "deemed less useful"?

A completely different issue that came up during my testing was this:
RIP, R.I.P, or 'rest in peace': How should that be analyzed? How would a human analyze it? It is of course a negative thing that a person has died, but rest in peace is a nice thing you say.

So, a simple short tweet like: "RIP Patrick Swayze", should that be analyzed as expressing positive or negative sentiment?
What do you think?

Is this the hot topic now in Web 3.0 - Sentiments on RealTime

One more to the list ..

Insttant finds real time updates to compile detailed analytics including sentiment,
user analysis and real time headlines, Insttant generates unfiltered,
real time news based upon individuals as well

I'm convinced! :)

Hi Alexandra,

I am grateful for your feedback on TipTop.  Your understanding is perfect.  This is an extremely hard problem and no existing solution comes anywhere close to perfection.  I would argue that even humans would not agree in many cases on what should be positive or negative.  (cf., the "RIP Patrick Swayze" question your raise at the end of your note.).  I would like to make only a few general points.

1.  Accuracy of the TipTop semantics engine is measured by us through two means:

(a) anecdotal evidence - people try it and give feedback.  Positive opinions have so far far outnumbered the negative ones.  In fact this forum is the first forum I have come across on the web where you and one other person have formed and expressed negative sentiment about TipTop's quality.

(b) We run through a large number of random selected queries and then evaluate the results.  The accuracy numbers are very impressive.

2.  To see sentiment about a specific concept or category, you have to click on it.  The overall sentiment for the query is only an approximation which we show to make the user experience a bit simpler.  Asking users to click twice before they see any interesting results would not be good user experience.

3.  TipTop search experience is about a lot more than sentiment analysis.  To really understand what TipTop is and will be, you need to play around with it a lot more.  Use all the features and then see what all magical abilities it has (and many more it will have as we are working 24/7 to add more cool features).  The value of some features won't be obvious to typical users for days and perhaps even weeks.

4.  There are lots of feeble attempts at Semantic analysis that end up drawing quite a bit of hype.  TipTop, in contrast, is based on 20 plus years of highly sophisticated research.  In the background there is both academic rigor as well as adequate practical understanding of the search industry.  I hope this comes through in the product.

I am happy to chat with you or anyone else who is interested in finding out more.

Thanks again.

Shyamkapur, I agree there are probably a lot of actors out there with semantic analysis tools that are less rigorous and serious - regarding both the research and the size and quality of the test corpora they base their analysis systems on, than other.. Everyone wants a piece of the cake.

OpenAmplify, develops natural language processing and text analysis software. Behind our core technology lies over 20 years of academic research. The company has been operating since 1999, I myself will in January celebrate my tenth anniversary  :)

It is nice to hear that we're not the only ones trying to do this properly, and we do enjoy a bit of competition.

I encourage you to have a look around on our site, see also openamplify.com, and of course: grab an API and test it!

Has this been analyzed for sentiment? Just curious as to the output of this.

Innovate or spend?  They are attacking the search market with so much cash in hand to spend, which is why their market share keeps growing.

Although they recently lost a couple % of market share: http://erictric.com/online/bing-us-market-share-dips-2-from-october-to-november

I really think Bing is going to have to innovate to overcome Google's user base advantage.

It means that Bing is going to be one of two major players in the search space.  Install that Bing Greasemonkey script now, before it becomes a "non unbundleable" piece of IE 9.