You come up with something that no one makes You are awesome Piyush.
@CuriousAnonDev
9 ай бұрын
Hey you are doing great work! A request- Can you please continue such tutorials and also teach about scalability, microservices, chat servers with rooms, video calls or deployment on docker-k8s etc. Like what software engineering looks like irl. People on yt just doing nextjs stuff and I can't understand anything...
@shravan2891
9 ай бұрын
Fresh unique stuff, no one is teaching this on KZitem. Also can you teach more about tueborepo in detail like testing, linting etc in a turborepo
@abdussamad0348
7 ай бұрын
Amazing tutorial, helping us to do better engineering. Industry level standards !!
@mdmusaddique_cse7458
2 ай бұрын
Man! So much of learning in these 2 videos. Thanks!
@debasishdutta9073
9 ай бұрын
Thanks piyush, I was waiting for this video. I love your scalable, system design videos
@souravkumar3553
5 ай бұрын
Really something awesome.Practically answering all system design question.
@niraz9701
9 ай бұрын
Please make a full video about rabbitMQ🙏
@shubhtalk7073
19 күн бұрын
You are best piyush bhaiya ❤
@Aitool-r3q
6 ай бұрын
Finally, completed this project and gained a lot of knowledge Thank you Piyush Sir ❤️👍🎉
@vikashgupta3305
7 күн бұрын
do provide a GH link.
@aadarshgurug
9 ай бұрын
that is something i was planning to build and had lot of confusion, now everything is cleared thank you brother
@TechSpot56
6 ай бұрын
Great video, please continue this series.
@akash-kumar737
Ай бұрын
Thank man. Learned a lot. Love you ❤❤❤
@sagar7929
9 ай бұрын
awesome course and thank you so much. Make more awesome valuable content with monorepo architecture in nodejs . God bless you sir and thank you once again.
@PrantikNoor
9 ай бұрын
I like your teaching style. ❤
@snehasish-bhuin
9 ай бұрын
Very nice learning ❤🎉it will definitely impact on community
@shubhtalk7073
19 күн бұрын
Better than paid course ❤
@rajukadel1007
8 ай бұрын
Great content !! Keep sharing your experience ❤
@abhidevlops
9 ай бұрын
Awessome Content Brother . You can further extend this Project ❤
@work7066
9 ай бұрын
why are we using redis along with kafka, can't we simply use Kafka's pub/sub for two servers to communicate instead of redis? can someone please explain the advantages or tradeoffs of doing so?
@AryanRaj-td4vs
7 ай бұрын
I'll give example based on Google pubsub or storage queue that I use at place of kafkaa, and the reason is if one consumer takes that message and ACK that message is gone from topic and other instances with same subscriber won't get the messages, offcourse we can create separate subscriber for each instance but that is manual process unlike redis pub sub
@swarajgandhi
Ай бұрын
Very helpful videos Piyush. Just have one doubt, How to retrieve data in case user refresh the page and we have to fetch the last few messages? because we can't query DB in that case.
@krishangopal2475
8 ай бұрын
Hi Piyush. Thanks for the amazing video!!! Just one Question, Couldn't we use kafka directly as a pub/sub instead of using redis separately , where all servers and the processing server ( running write queries in postgres ) subscribes to the 'MESSAGES' kafka topic?
@mohammedgazi786
9 ай бұрын
ek MVC pe bhi video lao bhai
@lakshyasyal
6 ай бұрын
This is a beneficial video but I didn't like the ending of this. With the try catch if the DB crashes or something goes wrong with it then pause for 1 minute and restart it from the beginning, Can we do something else that would be good for the DB always?
@FarazAhmad-t6g
3 ай бұрын
bro is litrally creating his own empire in backend mastery.
You are making a new entry to the database every time a new message is produced and as we know databases have low through put so can we run consumer or a second consumer to consume the data at a certain interval of time to store the produced data in db?
@anassaif3181
4 ай бұрын
You earlier said that consumer is a separate NodeJS Server but u defined the consumer in the primary server itself.. Why ? Is that for the sake of simplicity? If so , then how wil l get the same Prisma instance if we had standalone consumer server ?
@opsingh861
3 ай бұрын
You can just create the new instance for the prisma, and if you are using turbo repo then you can just import that
@anassaif3181
3 ай бұрын
@@opsingh861 New instance would probably lead to new connection ig and probably new migrations.... Yeah !, Turbo might be a better option.
@sohanbafna2282
4 ай бұрын
Here I was curious to know that if Redis could have been replaced kafka? I am not sure if redis is required here if we are using kafka ? Please let me know your thoughts
@shubhasheeshkundu6040
9 ай бұрын
Sir please make backend project using microservices
@Dark-nt8hh
Ай бұрын
One thing i didnt understand is why did you run the consumer function in init function of index.ts file. Couldn't understand the logic behind it. Everything else is topnotch
@skzahirulislam2820
9 ай бұрын
Bhai ek request hai plz ek pagination aur inifinite scroll ke upar bhi video banaye bahut jada ye topic pucha jata hai interviews mein react.js, node.js mein. Aur konsa kab use karna hai wo bhi bata dijiyega indepth banaye ga bhai
@AdityaGhode-l2z
3 ай бұрын
I had one doubt please anyone help, Where is the complete frontend and deployment part of this project ?
@wahabkazmi7486
Ай бұрын
Piyush you are sending data into the database one by one not in bulk, right?
@krishnanand_yadav
Ай бұрын
When the messages are consumed, then they should be deleted from the 'Topics' in Kafka, right? But they are still present there. Is it supposed to be like this?
@harsh_dagar
6 ай бұрын
Hey @piyushgargdev, Is Kafka's consumer interval (to consume message) is incremental or all data will be provided? If so, how can we only handle the incremental data and not whole?
@physicsakhada592
7 ай бұрын
If it is a group chat then we have to make a different database postgres to store all the data like room id and all users of that grp
@shi-nee7966
9 ай бұрын
where is the video piyush sir has mentioned about i.e the first part to this video?
@Sagar-b3q
9 ай бұрын
One more like... hey mate have query how kafka understands that to which consumer beed to send reply.. i did see your kafka video... struggling to understand this...and how can i build same for mobile app??
@indentCoding
9 ай бұрын
A doubt, You are able to get messages in kafka at high velocity because kafka is meant for that , but when you are inserting into db for eachMessage , how will it make any diff because in event of high velocity of messages eachMessage function will do a insert query into db so for example if you are rec 100000 messages at 1-2 second interval your db will have 100000 insert operation which will make the db down. and if that happen what is the benefit of using kafka i understand that there will be no downtime because kafka will still be active but there should be something that will reduce the insert operation into the db
@xcoder420
7 ай бұрын
Actually the consumer should be an altogether different microservice. This will consume the messages in batches and do batch insertion. Let's say we configured the DB to support 10k WPS. So we'll consume 10k messages and insert it into DB. This is actually known as async processing.
@Support-Phalestine
9 ай бұрын
Piyush ek introduction video banao plzz What is turboRepo im confused with it
@Chanchal_Sen
2 ай бұрын
You Got New Subscriber
@PrashantSaini-i6j
8 ай бұрын
Keep posting such content please
@akhiltej7q8
8 ай бұрын
please make more videos for this as continuation !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
@nitinjain4519
8 ай бұрын
Can i run two databases in prisma in the same project like PostgreSQL and MySQL
@sachinsingh2104
9 ай бұрын
can you help us in knowing how to deploy monorepo appilcations .
@digitalTechspace
6 ай бұрын
why you use redis and kafka both can we use kafka only?
@saksham_1612
9 ай бұрын
Awesome video ❤🎉
@deezwhat2791
9 ай бұрын
top notch content
@arafatislam4648
8 ай бұрын
Can you show the deployment process?
@jbrocksfellas4147
9 ай бұрын
awesome value
@aliimranadil2
9 ай бұрын
love it bhai
@growwithriu
9 ай бұрын
Hi Piyush and everyone, I have a doubt. When there are multiple servers, each of them will be consuming msg from kafka and writing to postgres, thereby creating as many message entries in db on every message as the number of servers. Is that desired? spin up one more server on a diff port. Send a message. There will be two entries for this message on db
@shantanubhise9288
9 ай бұрын
Yes, you are correct. If the logic for consuming messages and writing to PostgreSQL is directly placed within the message-receiving event, it can lead to duplicated message entries. In my implementation, I've used Redis for inter-server communication. I have not used kafka yet. When a message is sent (triggered by the "send" event), I publish it to the "MESSAGES" channel in Redis. And, on the "receive" event, I broadcast the message to all connected clients. Regarding the storage of messages in PostgreSQL, I've introduced a global array named "messageBatch." When a message is sent ("send" event), I push the message into this array. The important aspect is the use of setInterval to periodically process this array(use copy of messageBatch and make messageBatch empty to store new messages), writing its contents to PostgreSQL. The data is successfully stored.
@anagnaikgaunekar9081
Ай бұрын
This is a valid issue. This wont occur if we put the produceMessageForKafka logic after publishing to redis instead of putting it on subscribe
@_PiyushKumar-vv7ui
9 ай бұрын
amazing video
@rohitpandey4411
8 ай бұрын
Kafka itself have a pub/sub model, rather than saving data in two places(redis and kafka), can we create a data aggregation function that'll cater the user messaging service? Working adjacent to yours for updating the db update query handling function
@amardeepsingh1168
8 ай бұрын
We should use either Kafka or redis , right ? not both @rohitpandey4411
@iUmerFarooq
9 ай бұрын
How to handle real time notification in Vue&Node like FB handle its post notification. For example I have a Assignment Management system and I'm logged in as a Admin, when someone upload/send new assignment so the notification come in realtime and show in toast. How it can be possible?
@bilalddev-ew6gj
2 ай бұрын
love you.
@sagarbhatt3346
9 ай бұрын
please make the same videos on python.
@Ankit-01-01
9 ай бұрын
Make a video on posgresql
@tanvir4748
9 ай бұрын
Can anyone give me the previous video link?
@HimanshuDhiman-v9c
7 ай бұрын
can i do this with mongo ???
@catchroniclesbyanik
9 ай бұрын
I have one question. Since you introduced Kafka in the project, couldn't we remove Redis from the project?? Because Redis was being used for pub/sub, which Kafka can do as well.
@abdulrehmanjaved-rt8jq
9 ай бұрын
you mean we can subscribe on servers to kafka topics?
@abdulrehmanjaved-rt8jq
9 ай бұрын
Redis supports push-based delivery of messages that means messages published to Redis will be delivered automatically to subscribers immediately but kafka is supports pull-based delivery of messages, meaning that messages published in Kafka are never distributed directly to consumers, consumers subscribe to topics and ask for messages when consumers are ready to deal with them.
@shantanubhise9288
9 ай бұрын
Yes, one might think of using only Kafka or Redis. But here we need both. Here we have 2 requirements: 1. Inter-Server Communication. Meaning message sent by user1 on server1 should be received by all the users present on different servers. Here we can use Redis Pub/Sub model. Redis Publisher publishes the message to the channel "MESSAGES". All the Redis Subscribers of this channel will receive the message, including the server which has sent the message. Thus Inter-Server Communication is achieved. If we use Kafka in this case. Kafka producer will produce the message to topic "MESSAGES". Here all the Kafka consumers (on all servers) will belong to the same consumer group, because they have same groupId. Hence only any one consumer will receive the message on the "MESSAGES" topic. And other Kafka consumers (servers) will not receive the message. 2. Storage of Messages in Database. Here we can use Kafka. Kafka producer will produce the message to topic "MESSAGES". And only one Kafka consumer of this topic will receive the message. This consumer will store it in the database. Like I said earlier, all the consumers here have same groupId. Hence only one of them can receive message. If we use Redis here, all the Redis subscribers will receive the messages and store the messages in database, resulting in duplicate messages.
@catchroniclesbyanik
8 ай бұрын
Here, all the server instances subscribe to a redis channel for incoming messages. I think, we could simply remove redis, and make every server long poll kafka for messages.
@abdulrehmanjaved-rt8jq
8 ай бұрын
@@catchroniclesbyanikyes because kafka also gives pub/sub mechanism. I think pyush bhai ny first video just problem solve krny k liye bnai and yeh full scalability k liye.
@FaisalKhan-oy4zz
9 ай бұрын
when db is down or not able to insert that message so the consumer will resume from that message or from the next message?
@skzahirulislam2820
8 ай бұрын
That message because the message is stored inside kafka
@mystic_monk55
9 ай бұрын
👏👏
@ankitkapoordirector6087
9 ай бұрын
Please reply anyone can i make chat app using Java Networking concept? is it possible ? please reply
@xcoder420
7 ай бұрын
Yes. Explore about Netty.
@rishiraj2548
9 ай бұрын
🙏👍
@effectsofmp3927
9 ай бұрын
First comment 😁
@Sagarkun
7 ай бұрын
maza ni aaya bro
@thebigdanktheory6877
7 ай бұрын
Is there any way we can create virtual load to test our application? Then I can die peacefully 🎉
@vishalkhoje
8 ай бұрын
Hi Great work Piyush. Can you please create a video how we can able to deploy turborepo project like current scalable Realtime chat app on servers (vercel)
@ARSHADKHAN-hc6pb
9 ай бұрын
Love you brother ❤❤❤, And my one is that please make series on microservices project, that how to make project using this architecture
@riyanshbiswas
9 ай бұрын
Just one question, why are you using cloud services for postgres and kafka? Isn't using a docker container locally free and less time consuming as well?
@aniruddhadas2953
9 ай бұрын
Just wow 🤩 🤩 . I have learned something new that no one teaches us. Highly appreciable work. Thank you . 🙏
@harshgautam6260
9 ай бұрын
Mann i love how professionally you do your work ❤
@RiskyMeal
7 ай бұрын
Video Title In English... Video audio in Hindi... No offense... but Bruh... What are you doing?
Пікірлер: 92