How you speak is perfectly paced and you hardly stutter/'um'. I'm always trying to improve my speaking abilities and you offer a great example to follow. Keep it up man
@aemmelpear5788
3 жыл бұрын
I absolutely love how you can tell, how much you are excited about the Rust programming language. Always with a big beaming smile when talking about the nice features Rust has.
@nickmaxwellambient6615
4 жыл бұрын
Jon, just found your channel today. It's rare to find someone who both covers really in-depth topics and does so in an entertaining way. Fabulous job you are doing here.
@plorio
4 жыл бұрын
I used that exact same atomic counter technique to decide when to the other copy of data is safe to write to at work. Felt quite clever when I got it to work. Really cool to see the same thing in the wild.
@karlwhitford7523
2 жыл бұрын
Well done and don’t forget to debug
@distrologic2925
5 жыл бұрын
21:36 This is actually one of the best explanations of the rust ownership system I have heard yet.
@juliavanderkris5156
4 жыл бұрын
Damn yeah, that was great. I'm starting with learning Rust and the concept of borrowing seemed cool but also really confusing. This explanation made it so much easier to understand.
@gloatsticks
4 жыл бұрын
I'm slowly but surely learning Borrowing. The compiler is one of the most helpful with error messages!
@someonlinevideos
2 жыл бұрын
@@gloatsticks For some type T you have: Here is the text from that slide because Jon really does create a wonderfully succinct explanation of ownership in less words than this comment has used. T, &mut T, &T T is owned, &mut T is exclusive, &T is shared
@davidkeller6334
2 жыл бұрын
That explanation should be "borrowed" by every educator/communicator who wants to explain ownership
@morisonjazaj7920
2 ай бұрын
@@gloatsticks6
@yashashav_dk3766
4 жыл бұрын
This talk is worth it's weight in gold. Thank you kind, Jon!
@simonray4713
5 жыл бұрын
I pretty sure he love this language. Full of energy with a great talk
@rajsahu4644
Жыл бұрын
Jon is inspiring in so many ways. The work (which he explains in the presentation) and the way he described every topic! Just awesome. I probably never watched any hour long talk in one go until this one!
@scriptozavr
5 жыл бұрын
Enjoyed the talk, Jon is indeed a phenomenal speaker!
@yurikhomyakov9826
4 жыл бұрын
Thank you for an amazing talk. As soon as Rust 1.0 came out I knew something interesting was cooking!
@JohnHAdams-vo2pk
Жыл бұрын
Brilliant explanation of ownership. Simple and to the point. Better than any book I've read. Thank you for this, now it's pretty solid in my head.
4 жыл бұрын
This talk is amazing! You explain things in a very approachable way!
@sgdfdsgs
Жыл бұрын
Your talk was fantastic! Your passion and delivery were both amazing, and it really got me excited about Rust. Thank you for sharing your knowledge and enthusiasm with us.
5 жыл бұрын
Brilliant talk, thanks a lot! I really like how you've covered the ownership model, along with how you've implemented the lock free evmap datastructure and why that is safe. Well done.
@mahmoudabuzamel7038
4 жыл бұрын
Thank you very much Jon, this is a valuable presentation rich with guidance and recommendations. I learned a lot from it!
@ozank7327
5 жыл бұрын
great talk, rust looks amazing
@BohdanTrotsenko
Жыл бұрын
33:30 the datastructure suggested implements a "weaker map", meaning that writes are not seen.
@_akshaydeep
5 жыл бұрын
Database talk start at 13:40
@VishalAnand24
5 жыл бұрын
Don't jump, this talk is awesome
@eMbry00s
5 жыл бұрын
@@VishalAnand24 if you're into Rust, though, you've already been introduced to the Rust basics in like fifty other talks. So the timestamp is useful for some of us!
@VishalAnand24
5 жыл бұрын
@@eMbry00s okay
@prestigek1ngs
3 жыл бұрын
I’m currently getting into rust with no basically no prior knowledge. It’s kind of a pain but it’s been fun. I’m on the fence still because I can’t find the best tutorials but I’m trying hard. I think it’s the future
@adesanoyesamson668
2 жыл бұрын
2022 and I find this talk interesting
@sanderbos4243
2 жыл бұрын
I adored this talk, thank you!
@steveharris3627
4 жыл бұрын
Nice, sort of an advanced/smart “refresh ahead”, like caching systems do, that is built into the DB.
@distrologic2925
5 жыл бұрын
37:03 Instead of counting each read begin/finish, why not just keep a boolean value, which indicates whether the thread is currently reading or not. When starting a read it will set it to true, after finishing a read, it sets it back to false.
@jonhoo
5 жыл бұрын
If you did that, you might continuously read that readers are active if reads are sufficiently frequent and fast. That would block switching to the other map, even though it's safe to do so the moment all readers have finished at least one read.
@user-kn4wt
2 ай бұрын
@40:47 is that kdb with the benchmarking clause? 😅
@shirshak6738
5 жыл бұрын
thank you ! Looking forward for your new stream. Rust is the best programming language i have ever used :)
@TimLF
5 жыл бұрын
I was expecting "Multiversion concurrency control" of PostgreSQL, and CockroachDB to be compared, but then the video was over. I'll have to add that to my needs more research or testing list.
@joshuayanovski5765
4 жыл бұрын
MySQL also uses MVCC (albeit a slightly different variant of it, but one that actually makes reads by primary keys faster than Postgres at the cost of reads through secondary keys). The thing he's leaving out of this talk is that Noria does not fully support transactions, so it is not quite apples to apples; however, it is plausible that the queries he's testing are nonetheless transaction-safe (or have semantics that don't require full transactional isolation).
@thomascorbin5371
4 жыл бұрын
This was a fantastic talk
@darkenblade986
2 жыл бұрын
awesome stuff. I learned so much
@linuxsir-tw
5 жыл бұрын
Great talk, many thanks for your sharing
@passermelon7
5 жыл бұрын
I have a question...When you exchange the reader/writer pointer, you need to ensure all the reader pointer number are even.What if you check the latter pointer, the former pointer's number change.It seems like we still need a mutex?
@jonhoo
5 жыл бұрын
Ah, no, it's sufficient to observe an even number for each reader _separately_. For any given reader, if you observe their epoch number being even, you know that _that_ reader has seen the updated pointer, and so you don't need to check it again until the next time you want to swap the pointer.
@magnum789
4 жыл бұрын
@@jonhoo Hey Jon, very exciting talk! I'm new to Rust but have experience with Go. How does the writer reapply the writes it has done to map B into map A after swapping the pointers and revealing map B to readers and thus obtaining exclusive access to A? It seems like the writer must internally keep track of the missing writes (surely more than one missing write in map A due to possibly a reader being slower than the writer)? Another question: if the writer observes the even numbers separately for each reader, how does it prevent a reader from starting a long read just after the writer made the observations and just before it does the swap? Edit: I think I got the answers from reading the evmap docs now. There is a log, of course. The swap is just done at some unspecified point in time and the replay is done AFTER waiting for the epoches to get even. I'm thinking of trying to implement something like this in Go.
@jonhoo
4 жыл бұрын
Hey there! Almost missed this comment since it's under a relatively old one, and KZitem tends to hide those. Looks like you figured it out though! There is indeed an operation log so that the writer knows how to bring the "old" reader map up to speed when it becomes the "new" writer map. For the epochs, the way it works is that the writer will not start writing to what used to be the reader map until it has observed an even counter from every reader _since_ the swap. That ensures that any subsequent reader operation *must* be on the swapped map, and therefore there cannot be any readers left in the old reader map.
@junaid1464
Жыл бұрын
Can't postgres function idx do that?
@giganooz
Жыл бұрын
As for the database thing, how does it work when you have multiple queries that try to change data through addition at once. Wouldn't it read the same value multiple times first an then change that multiple times instead of going in parallel?
@jonabirdd
5 жыл бұрын
how can I upvote this talk twice?
@KhoaNguyen96
5 жыл бұрын
Create another account ... perhaps :)
@kompeterPC
5 жыл бұрын
Really interesting talk, well done
@KFlorent13
2 жыл бұрын
Thas was really really good.
@LaurentLaborde
3 жыл бұрын
i came here mostly for the database part of the talk :)
@arhyth
5 жыл бұрын
good stuff and great presentation!
@haydarinda
2 жыл бұрын
Thanks. Additionally, 0.8 is a reasonable speed for non-native speakers.
@serejkus
5 жыл бұрын
Great talk, thank you!
@mykra2939
5 жыл бұрын
Thanks a lot, great talk
@SahilDawka
5 жыл бұрын
You are a genius! Thank you!
@dbaldwin2803
Жыл бұрын
Does anyone know if this has been abandoned? The GitHub repo hasn’t been updated in quite some time and I really want this to flourish.
@jonhoo
Жыл бұрын
For Noria, the thing to watch is readyset.io/. For the data structure described in the talk, check out github.com/jonhoo/left-right/ !
@dbaldwin2803
Жыл бұрын
@@jonhoo so some of it is open-source and some is proprietary?
@jonhoo
Жыл бұрын
@@dbaldwin2803 All of Noria that was part of my research is open source on GitHub. However, since I graduated, it's not an active project, just the final state of the research prototype. ReadySet is building on top of the Noria code base and making it production ready. Some of that is proprietary, although much of it is also developed in the open over at github.com/readysettech/readyset
@dbaldwin2803
Жыл бұрын
@@jonhoo can you elaborate on which parts are proprietary? I tried pitching Datomic to our CTO for a new product and it being proprietary was a non-starter.
@jonhoo
Жыл бұрын
You'd be best off reaching out to them directly! While I'm a co-founder, I'm not involved with any of the ongoing work or plans :)
@colinmaharaj
2 жыл бұрын
7:56 hmmm, I rely on null pointers in C++ to know about the state of my objects.
@emdeization
2 жыл бұрын
Amazing
@micknamens8659
4 жыл бұрын
What happens when a read client on the partial materialized view A has an odd number of points in his access accounting, and the associated client thread died or is in an endless loop and will never end its query, i.e. won't increment its counter?
@jonhoo
4 жыл бұрын
It's a good question! The current implementation has no story for that - it would cause the writer to wait indefinitely. If you use Rust with unwinding, the trick to pull would be a "panic guard": a type created just before the user code is called that, if dropped during a panic, will increment the epoch counter to restore parity. If you want to try to submit a PR to evmap with that change, I'd be happy to mentor and review it! If the reader runs indefinitely, that requires the writer to wait as well.
@micknamens8659
4 жыл бұрын
@@jonhoo I'm currently just "interested" in Rust, so no PR from me 😉 . What about a time-out for Read handles. It expires after x seconds. That's then part of the API contract. Could be a semantic extension of Rust language or a user-defined wrapper type: when the read client tries to call a function of this type after expiry time an exception is raised on client side. So the writer need not wait for an expired read handle.
@jonhoo
4 жыл бұрын
@@micknamens8659 Hehe, that's okay! I think getting the semantics of expiring read handles right would be quite a challenge. And especially because you'd need some way to preempt a running thread, which is far from trivial!
@jonhoo
4 жыл бұрын
Just implemented this in github.com/jonhoo/rust-evmap/commit/73a67292b5588d6109b6845f6e771d804a9ac906 !
@micknamens8659
4 жыл бұрын
@@jonhoo Another approach to overcome this exceptional situation could be to create a temporary third materialized view for writing.
@Ethan-gu9hm
5 жыл бұрын
The idea of caching query results cannot be that new or novel isn’t it? I recall that oracle db has the feature with the name “materialized view” for a long time now
@jonhoo
5 жыл бұрын
There's a little bit of discussion around this in response to www.reddit.com/r/rust/comments/acucrs/rust_at_speed_building_a_fast_concurrent_database/edbxqv8/. In general, materialized views in commercial relational databases are limited, both in terms of performance and flexibility. I'd recommend giving the Noria paper a read if you want more in-depth analysis, as we evaluate that there too!
@panstromek
5 жыл бұрын
The idea of it is just so straghtforward... Basically since I started working with DBs, the whole inefficiency of their work has been bugging me, especially considering many of them were built at times when computational resources were sparse. I just don't get it.. I am glad that someone just took this pretty logical idea and just built it. Great work and talk @@jonhoo ;)
@robheusd
2 жыл бұрын
If I'm correct, even before materialized views were added, queries, parsed queries and their result sets were chached and stored in the shared pool of the PGA.
@piyushkatariya1040
5 жыл бұрын
Old wine in a new bottle. This is very similar to following the even sourcing model with a materialized view (MV). or Simply use PostgreSQL 11 + CitusDB (distributed computing and sharding) + PipelineDB (continuous computation and MV)
@jonhoo
5 жыл бұрын
Noria is actually very different from both of your proposed systems, but this was a relatively high-level talk that didn't talk much about the actual contributions that Noria makes. If you're interested in a more technical discussion, I'd recommend giving the research paper from OSDI'18 a read: www.usenix.org/conference/osdi18/presentation/gjengset
@jawad9757
2 жыл бұрын
Interesting that you used the word similar
@batorshikh.baavgaikhuu
5 жыл бұрын
Interesting!
@Kengur8
Жыл бұрын
Tea leafs on the stack.
@maherkhalil007
4 жыл бұрын
Can you make tutorial about using rust + databases + web assembly to produce MVC software?
@uuu12343
Жыл бұрын
Rust did many things right... Except community management and basic PR
@thomfox871
2 жыл бұрын
Storing indices instead of reference counting is just hiding raw pointer semantics.
@liquidmetal718
2 жыл бұрын
38:00 hmm, sus....
@VioletGiraffe
5 жыл бұрын
So bottom line is you still can't write fast code that can be statically checked for safety? Basically, it diminishes the advantages of Rust by a whole lot in my view.
@joshuayanovski5765
4 жыл бұрын
The question is how complex the invariant you want the compiler to check is, and what form it takes. Rust's compiler is capable of quite sophisticated reasoning about reference lifetimes, mutability XOR aliasing, and thread safety, but in order to write a library like evmap you need something else: support for monotone structures and their associated invariants. There is a framework for this, partial commutive monoids (with some extra stuff), and it's the basis for the Iris framework, which we are currently using to prove both the soundness of Rust's core type system, and the safety of many libraries that use unsafe (including core primitives like Mutex). However, proving that libraries preserve such invariants correctly is very technically challenging, and requires a very sophisticated type system that it would be impractical to integrate with Rust at this time (Iris is written in Coq on top of a proof mode with many years of work put into it). And it certainly isn't the case that constraint like "must satisfy this crazy monotone invariant" can be automatically discharged; you effectively have to write a mechanized soundness proof for any functions that touch the data structure that uses unsafe, to show that they preserve the defined invariant. What's nice about Rust (vs. other languages) is how much of this kind of sophisticated logic you can actually safely encapsulate safely via the type system, not just the fact that it has a safe subset.
@Codeaholic1
4 жыл бұрын
Jon I really appreciate what you do. You inspired me to sit down and learn Rust. You've made a difference in my programming journey. Thanks
@allesarfint
2 жыл бұрын
Something tells me you liked the language
@wrong1029
Жыл бұрын
He's extremely based. Thanks to him I doubled my salary and fell in love with a programming language.
@johanhellberg9677
10 ай бұрын
Congratulations@@wrong1029 ! Good work. Would you like to share how that came about, and what made the previous difference?
@ZenTrickz
5 жыл бұрын
49:16 "... your program will not compile unless everything is documented. It's fantastic!" You know you are a Computer Scientist when... 🤣
@YoloMonstaaa
3 жыл бұрын
@Ramon Oliveira you'll get used to it :)
@ekrem_dincel
3 жыл бұрын
@Ramon Oliveira not really.
@MrGeissT
5 жыл бұрын
Well done, Jon! Fantastic talk and really interesting to hear more about what you're doing when you're not Rusting on KZitem!
@danielfielding1938
3 жыл бұрын
If he were creating a fairly normal relational DB in Rust, then it would be a cool way to see the advantages of Rust, but those advantages seem to be somewhat obscured by the unconventional design of his unusual database. In other words, maybe the radical design is what makes Noria cool, not Rust?
@sluongng
5 жыл бұрын
at 52:00 he was describing that he had PTSD from fighting the Compiler ^_^
@nextlifeonearth
4 жыл бұрын
Or Stockholm syndrome.
@jonwise3419
5 жыл бұрын
I have a difference experience with rapid prototyping. To me rapid prototyping is MUCH easier with a good powerful type system if you do it in the right way. I'm still learning Rust, so I'll use Haskell as an example, but I already see it will be the same in Rust. In Haskell rapid prototyping is a breeze. Remember, that you don't need to write implementations or correct code! In Haskell you can write "undefined" and compiler will shut up. I normally, have several shorthands like "u = undefined" and then write a type and `u` as an implementation, compile, write more code replacing `u` with a more concrete implementation (that still can have more `u`s inside it) and then slowly an implementation writes itself as you replace undefined parts of your code with more and more concrete implementations. I heard this being called as type-driven or hole-driven development, because you follow the type, put holes in your code and fill those holes. Also, remember that with type inference you can replace anything with `_` in Haskell or with a sham type in Rust (like u8 when you know something is not u8) to figure its type or what type you need to pass, because it will complain and show you exactly which type something needs. Rapid prototyping is all about speeding up the feedback cycle and code-reuse. The latter is about not writing code at all, because you already have a library somewhere. Both are enabled by a strong and flexible type system. Starting with feedback, in Python to detect that your code is not working you'll have to wait until runtime .In Haskell or Rust, you'll immediately get feedback the moment you press a key. In fact, I usually wrap a function into another function that will print some useful output and I have very fast key shortcuts to both compile and to show some runtime feedback as well (like to print a datastructure I'm working on or run just tests that matter, keeping it fast). Like there will usually be a predefined name for me in a file, like `c`, which I can use at any time when working on a file. If I want to print something I just put one line in a file like `c = my_function testArg` and this is what will start getting printed out as I'm working on that function because that `c` will be like a main for that file, while I'm working on it that will constantly print output, and a script will automatically import that file and run it as I'm working on it. Same for tracing statements, pretty printing, and other tools that you would normally use. I want all of them to be a short names that I can use anywhere as I'm rapidly typing and working on something, preferably as a Swiss knife personal library that you can import in a few keystrokes into any file. Using other libraries is easier in a languages with a good type system due to a) type inference b) faster feedback cycles c) confidence in that if it compiles it will work. So, you can again just put a bunch of undefines, type holes / sham types, and slowly figure out what do you really need to some interface of that library or how to get there. In Python you can pass invalid data and you'll have to wait until runtime to figure this out. In Haskell or Rust, you will get immediate feedback that something is wrong and you have type inference to figure out EXACTLY what data needs to be there or how it should be used, and by using type holes can try to arrive at that point in any way you want by splitting the problem into smaller parts and following the types. What Rust misses here is more libraries, but when it will have a lot of libraries, figuring out those libraries will be much easier for reasons I've mentioned. Prototyping in Python can be easier DESPITE the lack of type system because there are more libraries so a lot of code you don't have to write. But when I have to write it, it's more painful. Python also has some good tools to speed up feedback cycles, like IPython, so some people naturally use workflows with better feedback cycles. But REPL is really not necessary if you have a good workflow which imitates REPL automatically. In fact, REPL can''t MAXIMIZE speed. Like you don't want to be typing code in REPL and then also in your file. You want your file to automatically give you REPL-like feedback. There are several ways to do it, beyond having scrips and conventions to automatically run things. You can have one file that acts as a RELP, imports a library you are working on and gets continuously run as you work on that library. I just find that more direct approach is faster. I guess to summarize this, I would say that if you don't fight the compiler but optimize your workflow then rapid then prototyping is easier with a more powerful type system. Rust can have no type system at all if you don't want it to, because you can ignore any parts of the code with unimplemented or panic.
@blo0mfilter868
3 жыл бұрын
nice blog post
@QmVuamFtaW4
2 жыл бұрын
uhh can you explain all of it in under like 17 words? my brain hurts
@julians.2597
6 ай бұрын
@@QmVuamFtaW4 Grug like Rust. Rust compiler not quite REPL, but compiler know lots things. Compiler often tell Grug when mistake in thing Grug makes. Tells also where problem abd how to make mistake go away.
@corv882002
Жыл бұрын
Very interesting talk. I'm curious why you couldn't just store an atomic counter of how many readers are active per map instead of doing the whole epoch and ignore it if it's even thing.
@dsincl12
5 жыл бұрын
Really enjoyed this talk. The epiphany for me when hitting the borrow checker wall was when I realized that it was helping me not penalizing me for the code I've written. This small mental shift made everything all the easier and suddenly Rust became a pure joy to work with. I have to admit I didn't read through the whole Rust book from start to finish before hitting the wall. I usually never do that when learning a new language but you are absolutely right. Reading the book from start to finish will save you a lot of pain and frustration with the language.
@thisiswill
4 жыл бұрын
You’re good at explaining this - and I’m very new to exploring Rust. Thanks for posting this keynote.
@LiamDennehy
Жыл бұрын
I'm so glad i found this presentation again. I'm really getting into Rust and even advocating for it in my work, and recalled this presentation does a great job introducing Rust and proving how it can solve problems.
@flesnuk
5 жыл бұрын
Thank you for uploading this, the second part of the talk is awesome and easy to follow.
@thingsiplay
3 жыл бұрын
40 years from now, no one will understand how someone could even write unsafe code. You can quote me, if I am wrong.
@FrancescoCielo
5 жыл бұрын
Isn't storing nodes of a graph in a list and then using indices basically just doing pointers while hiding it from the borrow checker so it doesn't bother you?
@jonhoo
5 жыл бұрын
Hehe, yes, pretty much!
@Truth-p7c
20 сағат бұрын
You need to grow your hair back man!
@hanskessock3941
Жыл бұрын
Jon is a wonderful teacher, and inspirational.
@samferrer
2 жыл бұрын
Wonderful lecture ... and the switchable rw-lanes is a must takeaway... thank you!
@BenjaminCronce
5 жыл бұрын
Super-linear is real, up to a point. Good memory access patterns can cause different threads to share cached data, such that 2 threads is more than 2x faster than 1 and 4 threads is more than 2x faster than 2. But in my limited experience, I never noticed past 4-6 threads, but at least beyond that was essentially perfectly linear.
@bozhidaratanasov7800
Жыл бұрын
Do you think Rust has caught up to speed on async networking stuff? I am trying to follow the Rust trends, but I am not sure what the difference between now and 4 years ago is. Tokio seems to have been doing a fair bit of work there.
@vhiremath4
4 жыл бұрын
This was an incredible talk. Thank you. I shared this with my team - I'd love for us to consider using Rust for some lower-level video stuff. :-)
@yurizappo5726
3 жыл бұрын
Love the idea of double increment and check if number is even. Simple and effective.
@flyingsquirrel3271
5 жыл бұрын
Thanks for the very impressive and interesting talk! If I wouldn't love rust already, you would have convinced me to learn it ;)
@benoitranque7791
9 ай бұрын
This was fantastic, thank you
@ddystopia8091
Жыл бұрын
It is just incredible talk
@haystackdmilith
5 жыл бұрын
Great stuff
@randomplan9537
4 жыл бұрын
thank you for this great intro to noria. is there a way to try noria also ?
@jonhoo
4 жыл бұрын
github.com/mit-pdos/noria is what you'll want to look at :)
@creatcodebuild7180
5 жыл бұрын
Hi Jon, this is a great talk and I am reading your paper as well. I found this video while searching on the topic of Read/Write separation in databases because Datomic is not open sourced thus I can't study its implementation. Have you ever evaluated Datomic? If so, what are your thoughts of Datomic and how different it is from Noria?
@mxoliv_
6 ай бұрын
amazing talk!
@robheusd
2 жыл бұрын
Isn't there a chance that just after the writer sees that all read counts are even and it is safe to do a write, and just before the writer/reader pointers get interchanged bfore the actual write, a reader starts a read, or what prevents the reader from doing that?
@jonhoo
2 жыл бұрын
The writer swaps the pointer *first*, then waits for all the counters to change, and the starts modifying the old pointer. If a reader starts doing another read, they'd follow the swapped in pointer, which points to the other copy (the one the writer isn't about to modify)
@clinton9129
Жыл бұрын
Jon, thank you and your friends for digging deep and discovering the possibilities. Since our last discussion, I am now learning how to code in python and I am learning other languages also. But my idea behind RUST is low latency and Jupyter notebook style coding. In fact Jupyter notebook was also one of my crazy ideas. Now that I am learning how to code, my writings' will be phenomenal given my skills. Expect more movies, lyrics to songs, Comedy segments, etc. Anything is possible now as always.
@davidboreham
2 жыл бұрын
Quite surprised MySQL and Memcached are so fast by comparison. I'd expect more like O(10^2) difference.
@paolophoenix
5 жыл бұрын
We did reasearch on this topic 25 years ago. The approach fail on non monotone schemas. Sometimes you have to recompute all the db. Easy to demonstrate. I can write a db in wich if you add a record in a table (cache) that cause all other cache to empty. The future is in functional databases. Where tables are functions.
@jonhoo
5 жыл бұрын
It's true that there are cases like that, but the argument we're making here is that that is not true for *common* queries for web applications specifically. And even in those cases, if your application is read-heavy, it is far better to re-compute the query result only when the result changes, rather than on *every* read. If you have a particular problematic use-case in this domain in mind though, I'd be happy to take a look!
@colinmaharaj
2 жыл бұрын
16:40 That is spectacular, I like it, will try that. (Update the read caches on write)
@bobwatson957
3 жыл бұрын
Excellent description of some Rust features that will trip up the newb. Excellent video.
@ankurleonardo
2 жыл бұрын
Every second in this video was worth watching. Awesome DS glimpse.
@markocvejic6416
3 жыл бұрын
Link for voting on your youtube chanel, intro video is broken :)
@veramentegina
4 жыл бұрын
wow, what a great talk. Passion and enthusiasm makes such a difference. It captivated me. Thank you so much.
@MaxLambrecht
3 жыл бұрын
Amazing presentation. Now I love Rust more.
@chang8106
3 жыл бұрын
Thanks for sharing. Appreciate it
@MIHIRAMRELIA
2 жыл бұрын
Love your work, enjoyed the session !
@Euquila
5 жыл бұрын
i want to upvote but then i would be the 666th upvote
@DavidBeaumont
5 жыл бұрын
I'm interested in whether this approach is safe in shared memory multi-cpu systems. Aren't you relying on the order in which data pages are published to the different CPUs to be able to promise that an even count reflects the true state of the reader in another thread? Does Rust have some promises about the semantics of the memory architecture? (e.g. are you implicitly relying on the counter and the table pointers being in the same memory / same cache lines?)
@jonhoo
5 жыл бұрын
The epoch counters use atomic loads and stores (fech-add specifically) with well-defined memory orderings. Take a look at the AtomicUsize type in the Rust standard library :)
@raysonlogin
3 жыл бұрын
The 1 map for readers and another one for writer mechanism is like RCU - whenever the writer is ready then the pointer to the data structure is published such that the readers get the updated version.
@jonhoo
3 жыл бұрын
Yup, that's exactly right, except that in this scheme we only ever have two copies and switch between them. In RCU you create a new copy for each update.
@dan-vw4ve
4 жыл бұрын
Really awesome talk, one thing I don't get is the reasoning for an epoch counter instead of flipping between 0 and 1? edit: is it an ABA thing?
@jonhoo
4 жыл бұрын
So, you could get pretty far by just flipping between 0 and 1, but using an actual counter enables some cool optimizations. For example, in the real implementation, all writes are delayed until the _next_ swap, and only then are they applied to the "write" side of the map, which is then immediately made the "read" map and exposed to readers. This allows the swap to only block until the _previous_ pointer swap has been observed by all threads, at which point the "write" side is safe to modify. This, in turn, means that usually the swap doesn't have to wait _at_ _all_ , since in the time between the previous swap and the current swap, most reader threads will have finished any pending operation they had. _But_, this relies on being able to distinguish between a reader that is still doing the _same_ operation it was doing when the previous swap happened (and so hasn't seen the latest pointer swap) and one that is doing a _new_ operation (and so _has_ seen the latest pointer swap). And 0/1 wouldn't let you do that, so you would _always_ have to wait to observe a 0 from each thread.
@psychicopus
4 жыл бұрын
My inspiration
@guibirow
4 жыл бұрын
At 37:00, to check if there are no readers still reading or suspended before starting writing, he suggested to increment the counter twice. One on starting the read, the other when completed, then check if the counter is even before writing. With this approach, if there are two readers or any even number, will cause the writer to falsely assume there is no reader because the counter will be even. I would assume the right approach is increment on start and decrease when finished. Then, if the counter is 0, the writer can confidently guarantee no reader is pending finalization before start writing. Is there anything I am missing?
Пікірлер: 169