(no subject)
1. Все хвалили invited talk про interleaving = online evaluation двух функций ранжирования по пользовательскому поведению.
Abstract:
Online evaluation is amongst the few evaluation techniques available to the information retrieval community that is guaranteed to reflect how users actually respond to improvements developed by the community.
Broadly speaking, online evaluation refers to any evaluation of retrieval quality conducted while observing user behavior in a natural context. For example, in interleaving experiments, the rankings computed by two retrieval functions are interleaved together and then presented to the user. Afterward, clicks can be interpreted as a relative preference of one retrieval function over the other. The effectiveness of such approaches lie in their ability to directly measure how real users perceive retrieval quality, as well as their ability to preemptively control for various sources of bias or noise.
2. Была статья про SGD с оптимизацией параметра регуляризации на ходу: достаем пример из тренировочного множества, апдейтим параметры, затем достаем элемент из валидационного и апдейтим параметры регуляризации, минимизируя ошибку. Экспериментально становиться круто =)
Abstract:
Online evaluation is amongst the few evaluation techniques available to the information retrieval community that is guaranteed to reflect how users actually respond to improvements developed by the community.
Broadly speaking, online evaluation refers to any evaluation of retrieval quality conducted while observing user behavior in a natural context. For example, in interleaving experiments, the rankings computed by two retrieval functions are interleaved together and then presented to the user. Afterward, clicks can be interpreted as a relative preference of one retrieval function over the other. The effectiveness of such approaches lie in their ability to directly measure how real users perceive retrieval quality, as well as their ability to preemptively control for various sources of bias or noise.
2. Была статья про SGD с оптимизацией параметра регуляризации на ходу: достаем пример из тренировочного множества, апдейтим параметры, затем достаем элемент из валидационного и апдейтим параметры регуляризации, минимизируя ошибку. Экспериментально становиться круто =)