This was really insightful. Hadn't worked on DDB before and this video helped me understand how it works and what use cases I can use it for. Was wondering if you could come up with similar content for Cassandra which explains a hard-to-solve problem for Cassandra and gives an insight into the database itself?
@TheHp6515b
2 жыл бұрын
How do you ensure correctness ? Updates from lambda to ddb aren't idempotent iiuc so updates might be processed multiple times ? Also streams to lambda is at-least once and not atmost once so another reason for duplicates.
@vaibhavjain2123
2 жыл бұрын
Loved the idea of using serverless and optimizing for common use case (i.e. very less frequent operation) Gave some new ideas for sure while designing systems 😄
@pratyushprateek2934
Жыл бұрын
Instead of this, can’t we use Redis, with our key as restaurant ID and value as list of users who have liked the restaurant. Or we can maintain two type of KV stores, one storing exactly same as restaurantid_userid and other storing restaurantid vs number of users which have liked it. If we talk about the like request flow, we add the key restaurantid_userid in first KV store, and increment the count for restaurantid in the second KV store. Similarly, for the unlike request flow, we remove the restaurantid_userid from first KV store and decrement count for restaurantid in second KV store. First KV store can be partitioned by the same key restaurantid_userid, second one can be paritioned by restaurantid.
@AsliEngineering
Жыл бұрын
Yes we can use Redis or a blunt KV store.
@blunderfoxbeta
Жыл бұрын
@arpith is it good to keep DynamoDB as a primary database and elastic search as a secondary database for aggregation and search purpose?
@TheCosmique11
2 жыл бұрын
Given that it's only 7000 favourites per day wouldn't have been easier just to do synchronous update to aggregate fav table. I am suggesting this only because requests are only about 7000 per day. Would like to know your thoughts on this?
@sanjaybhosale8296
2 жыл бұрын
Liked the idea of streaming, but can't we create new index to store aggregated value and use transaction so that both operations will happen in sync. Increment/decrement you can do on new index
@akhileshsirohi7627
8 ай бұрын
I was thinking the same as well.
@joobis.b4568
Жыл бұрын
Really interesting ! As you said if we let the API itself update at 2 places problem is it would increase the latency, but can't we make the 2nd update asychronous (like via a goroutine in Go) from the API ? Will that be a good approach ?
@AsliEngineering
Жыл бұрын
We need guarantee that both happens What if process crashes after 1st but before 2nd?
@joobis.b4568
Жыл бұрын
@@AsliEngineering right got it !
@soumyaranjanpatel1346
2 жыл бұрын
@Arpit does the dynamoDB stream ever fail, or what could be done for making it fail safe.
@AsliEngineering
2 жыл бұрын
That is AWSs headache. They would be guaranteeing it
@utkarshdevganalterego
3 ай бұрын
Why didn't you use AWS kinesis instead of dynamodb streams?
@AsliEngineering
3 ай бұрын
This is what Doordash did, not me. I am staying true to the source. Several approaches to solve the same problem do exists and company choosing one of them is highly contextual.
@utsavprabhakar5072
Жыл бұрын
Hi Arpit, following you since the early times. You are doing a great job and i would love to meet you someday in bangalore? Anyway one doubt. If we have to show this data in a sorted manner, wont ddb be inefficient? Even after creating the new table?
@AsliEngineering
Жыл бұрын
Thanks a ton! With appropriate sort key and schema this can be handled. Just an extra index would suffice.
@utsavprabhakar5072
Жыл бұрын
@@AsliEngineering yes but was just confirming it in the current example. That wouldn't be possible right?
@akashshirale1927
2 жыл бұрын
Loved this video. So suppose we have shards of Dynamo DB, then each Shard will have its own queue..or they will have one common queue between them?
@sourabhkhandelwal689
2 жыл бұрын
I think each shard is uniquely identified by a partition key and entries within each partition are sorted based on sort key.
@akashshirale1927
2 жыл бұрын
Ya right thats how data is added to shards by the service…but like how is it consumed by the message queue..?
@sparsh724
2 жыл бұрын
@arpit if Aggregated favorites PK and SK are designed in this way won't it lead to hotkey issue during peak time?
@AsliEngineering
2 жыл бұрын
For this use case, no. What are the chances that million people would mark the same restaurant as favourite at the same time. highly unlikely.
Пікірлер: 25