Generate conclusion for a given text - guidance

[george-i]

Here is an example of what I'm trying to achieve.

The text:

Last year we had poor crops. Most likely because of the prolonged drought. This spring, however, it rained generously.

The expected output would be:

The crops will be rich this year.

My first thought on the workflow to generate a conclusion is this:

• break text into sentences
• identify the subject
• identify times in sentences
• sort sentences by times
• identify positives/negatives or sentiment in sentences
• ....what now?

I have a little experience with NLP and compromise even less.
So I don't even know if what I'm trying to achieve is possible, or if it can have a decent result.

Suggestions?

[spencermountain]

hey George, this is a really tricky problem and a good question.

To me, understanding a language requires a knowledge base, to already know things like crops are plants and plants need water.
Compromise doesn't offer this, but it does know that a crop is something, and a drought is something, so you can wire them up together in a clever way.

There are some compelling demonstrations for text-summarization, which is a lot of the frequency-based analysis that you described. But it's really just about plucking-out salient parts of a text. I don't know any projects for something like you described, with logical deductions. Maybe others know some related work, it would be very hard.

Please let me know if you get anywhere with this,
cheers

[thegoatherder]

@george-i This is more a problem for Cognitive Semantics than NLP. It’s a particularly hard problem space and there are many trying to tackle it with mixed results.

You might find machine learning models are a better place to start. Look for a tool with a Dependency Parser feature as this is key to the problem you are trying to solve. For example SpaCy offers one.

I would also suggest reading works by Ray Jackendoff and Stephen Pinker, among others, to familiarise yourself with the theory.

[thegoatherder]

PS - also consider that the size of the problem relates to the scope of the knowledge domain. So, if ALL of your example texts to process relate to crops and farming; then it is somewhat easier to solve and could potentially be done using NLP alone. If on the other hand you wanted to infer logical conclusions for, say, any Tweet posted on Twitter or a blog, about any subject - well then it becomes a mammoth undertaking.

[george-i]

Thank you @spencermountain and @thegoatherder, I understand where I'm standing now.
What I asked for seems to be the tip of the iceberg.

Until I get there I need to make other steps first.
One of them is to create an autosuggest, Gmail-like, when you compose an email.

I'm thinking that even though I don't generate a conclusion, I could start from completing phrases.

Would be helpful to use the chat-bot tutorials from compromise?