What is in a Lucene index? Adrien Grand, Software Engineer, Elasticsearch

Channel:LuceneSolrRevolution · Length:1:04:30 · Published:2013-12-15 · Views:80696

Watch on YouTube

Summary

Elasticsearch engineer and Lucene committer Adrien Grand explains the design decisions inside a Lucene index and what problems it can and cannot solve. The talk goes into how Lucene stores data for fast search and analytics.

Why HN recommends it

HN recommends it as an instructive, easy-to-follow look at Lucene internals from a core committer.

I can super highly recommend talks by Adrien Grand on Lucene and Elasticsearch. — HN, 2020

Databases & storage · Algorithms & data structures

Subtitles (excerpt)

[Music] we are going to talk about uh Lucin internals it's something that I found very interesting so I'm going to try to make it uh interesting for you too about me so my name is Adrian gr I come from France uh I'm a Lucan solar Comm and I I work at elastic search and one one way you can see that I'm interested in the losing internals that have changed a lot of the index file format over the last month so I started with start field but I ended up uh working with the St feed format the ter vector format and dog values as well and I'll so I'll try to give you insight into the the way it works uh with Lucin so one question maybe you had before coming to this talk is why should I learn about uh loen internals so you may think that if you need to to learn about the internals of something it means that that the API is not good enough but actually there are very good reason to learn about Lo internals the first good reason is to to know the cost of the apis so it's very important to know the cost of each API if you want to to best uh use them and to build blazing fast uh s applications uh there are a few things that you should never do with Lucin especially you should never commit all the time so doing doing a Lucin commit every

On Hacker News

I can super highly recommend talks by Adrien Grand on Lucene and Elasticsearch. They are super instructive yet easy to follow. [VIDEO] [LINK] — drej, 2020-01
This is excellent. I was recently reviewing Lucene concepts and found this video really good: [VIDEO] Also this site has a series of Lucene articles that are pretty nice. The one on Term Vectors in particular: [LINK] Based on some quick research it seems like Lucene is already using a sorted skip data structure for the posting list, so I wonder why they had to do a custom implementation? Perhaps … — stereosteve, 2020-06
This is extremely high level. A nice view of how lucene internals work by core committer Adrien Grand: [LINK] [VIDEO] — ddorian43, 2019-09

Related videos