Dataflow matrix machines as generalized recurrent neural networks

[anhinga_anhinga]

A year ago I posted about dataflow programming and linear models of computation:

http://anhinga-anhinga.livejournal.com/82757.html

It turns out that those dataflow matrix machines are a fairly powerful generalization of recurrent neural networks.

Comments

[livejournal]

Hello! Your entry got to top-25 of the most popular entries in LiveJournal!
Learn more about LiveJournal Ratings in FAQ.

[anhinga_anhinga]

I wonder if this strange "top-25" marking is real :-) (It is supposed to count the number of visits, but I doubt it can distinguish genuine visits from bots, and there was quite a bit of bot activity.)

In any case, I'll be happy to post further details/comments on DMMs and/or recurrent neural nets, if anyone wants them.

And, on the other note, I'll be happy to post Clojure notes, if anyone wants to see those. If anyone is potentially interested in learning Clojure (a JVM-based modern Lisp) or in improving their Clojure skills, those notes might come handy...

[bvn_mai]

Я думаю, что многие заходят молча :) - это не боты. Ну а что касается деталей - мне это интересно.

[bvn_mai]

Кстати, 5-ти кратное повторение моего комментария - это явное свидетельство, что это не боты а кривые руки, новых владельцев ЖЖ.

[anhinga_anhinga]

Да, глючит ужасно. Буду потихоньку писать серию комментариев к этому посту :-)

[anhinga_anhinga]

So, I'll try to write this as a series of informal comments

[anhinga_anhinga]

I am trying to keep two angles of view on this subject at the same time: neural nets as a computational platform and neural nets as a machine learning platform

[anhinga_anhinga]

The following post "The Unreasonable Effectiveness of Recurrent Neural Networks" by Andrej Karpathy illustrates the remarkable power of LSTM as a machine learning platform

[datjko]

Misha, thank you, looks very interesting but I'm rather a newbie here. Could you answer a first bunch of my silly questions here or give me a link to some introduction? I started learning ml more or less seriously only recently and some looking into the papers you gave makes it clear for me: it's yet another interesting story I started to listen from the middle of the book

[anhinga_anhinga]

Привет! Let me start with answering your questions, then pointing to some links

[datjko]

Привет!
Wow! So generic and still workable!
I'll try to read through and return with new questions. How nice to be able to get answers from first hands.
May be you could also write some post resolving some typical noob questions and difficulties in understanding? Say, assuming not bad undergraduate background in CS/ML/FP of the reader? I believe many people would greatly appreciate it.
Thank you, Misha!