The code demonstrated this video can be downloaded here: lightning.ai/lightning-ai/studios/statquest-introduction-to-coding-neural-networks-with-pytorch?view=public§ion=all To learn more about Lightning: lightning.ai/ This StatQuest assumes that you are already familiar with... Neural Networks: kzitem.info/news/bejne/pKeFzJ1qan6Xd6w Backpropagation: kzitem.info/news/bejne/qoRovqF4oXt9p2k The ReLU Activation Function: kzitem.info/news/bejne/l254wGmccIJqZ3o Tensors: kzitem.info/news/bejne/rWlrzHp6qal6gGk To install PyTorch see: pytorch.org/get-started/locally/ To install matplotlib, see: matplotlib.org/stable/users/getting_started/ To install seaborn, see: seaborn.pydata.org/installing.html Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
@yongjiewang9686
2 жыл бұрын
REALLY Hope you can continue with this PyTorch tutorial.
@statquest
2 жыл бұрын
@@yongjiewang9686 Will do!
@shichengguo8064
2 жыл бұрын
Do we have video talking about transformer? Thanks.
@statquest
2 жыл бұрын
@@shichengguo8064 Not yet, but soon.
@Mayur7Garg
2 жыл бұрын
Just a small comment. Any variable should not be named similar to any builtin in Python. The 'input' variable in forward should have been called something else since it is already a builtin function in Python. Otherwise, you end up overriding the builtin within that scope.
@ToyExamples
12 күн бұрын
The style of storytelling is just so unique and friendly
@statquest
12 күн бұрын
Thanks!
@santoshmohanram536
2 жыл бұрын
Favorite teacher with my favorite Deep learning framework. Lucky to have you. Thanks brother🙏
@statquest
2 жыл бұрын
Wow, thanks
@firesongs
2 жыл бұрын
Please continue to go through every single line of code including the parameters with excruciating detail like you do. None of my professors went over each line like that cuz they always "assumed we already knew" and everyone in the class who didnt already know was afraid to ask to avoid looking stupid. Thank you.
@statquest
2 жыл бұрын
Thanks! Will do!
@insushin6139
7 ай бұрын
StatQuest is the GOAT in statistics, machine learning, and deep learning! You're videos are really helping me understanding the concepts and outline of these fields! Love from Korea!
@statquest
7 ай бұрын
Thank you!
@youlahr7589
2 жыл бұрын
Ive used PyTorch for projects before, but I can honestly say that I never fully understood the workings of building a model. I knew that i needed the peices you mentioned, but not why I needed them. You've just explained it incredibly. Please don't stop making this series!!
@statquest
2 жыл бұрын
Thank you very much! :)
@MugIce-lr6ui
3 ай бұрын
Hello! Not sure if anyone's pointed this out yet, but the code on 10:14, 12:09, and 22:42 needs a small addition, `plt.show()`, or else it won't show the graph. Though, maybe 2 years ago when this video was made you didn't need that, I'm not sure, haha. Other than that, this is an awesome tutorial that quite literally takes anyone through the process step-by-step, and even tells you some neat fun facts (like the sns nickname) and explanations like how `loss.backward()` works. TRIPLE BAM indeed! Thanks for the awesome tutorials and videos you put out 👍
@statquest
3 ай бұрын
Thanks! Did you run my code or type it in yourself? I keep the jupyter notebook updated.
@ni3d4888
Ай бұрын
plt.show() helped me get the visualizations in Ubuntu under WSL on Windows 11. Thank you for the comment.
@footballistaedit25
2 жыл бұрын
Thanks for the best content you bring. I hope you continue to make a full pytorch playlist
@statquest
2 жыл бұрын
That's the plan!
@footballistaedit25
2 жыл бұрын
@@statquest Thank you so much
@gummybear8883
2 жыл бұрын
What a blessing this is. You are indeed the Richard Feynman of Data Science.
@statquest
2 жыл бұрын
Thank you!
@jonnhw
2 жыл бұрын
Was looking for a pytorch resource and was disappointed when this channel didnt have one yet but then this got uploaded. Really a blessing to the people haha
@statquest
2 жыл бұрын
Thanks!
@Nonexistent_007
20 күн бұрын
Thank you sir. You have no idea how valuable and helpful your videos are. Keep this good work running
@statquest
20 күн бұрын
Thanks, will do!
@karlnikolasalcala8208
11 ай бұрын
YOU ARE THE BEST TEACHER EVER JOSHH!! I wish you can feel the raw feeling we feel when we watch your videos
@statquest
11 ай бұрын
Thank you?
@viveksundaram4420
2 жыл бұрын
Man, you are love. I started my neural net journey from your videos and it's the best decision I made. Thank you
@statquest
2 жыл бұрын
Hooray!
@binhle9475
Жыл бұрын
AMAZING video. This is exactly what beginners need to start the Pytorch journey with a semi solid footing instead of mindless copying. Yoy must have spent so much time for your AWESOME videos. GREATLY appreciate your effort. Keep up the good work.
@statquest
Жыл бұрын
Thank you very much! :)
@jamilahmed2926
Жыл бұрын
I have lived long enough to watch videos and understand nothing about ML stuffs, until I saw your videos. I truly wish your well being
@statquest
Жыл бұрын
Thank you!
@AlbertsJohann
2 жыл бұрын
What a great feeling when it all clicks after learning about all these concepts in isolation. All thanks to an incredibly brilliant teacher! Triple BAM!!!
@statquest
2 жыл бұрын
Hooray!!! Thank you!
@kaanzt
Жыл бұрын
That's really cool explanation! Please continue this PyTorch series, we really need it. BAM!
@statquest
Жыл бұрын
Will do!
@sapnasharma4476
2 жыл бұрын
Thanks for the awesome tutorial! You make the most difficult things so easy to understand, specially with the visuals and the arrows and all! The comments written on the right hand side make it so more helpful to pause and absorb. I would never miss a video of your tutorials!
@statquest
2 жыл бұрын
Hooray! I'm glad you like my videos. :)
@frederikschnebel2977
2 жыл бұрын
Thanks so much for this gem John! Literally got a PyTorch project coming up and your timing is just perfect. Greatly appreciate the content, keep up the good work :)
@statquest
2 жыл бұрын
Thank you!
@anashaat95
Жыл бұрын
This series about neural networks and deep learning is very well explained. Thank you soooooooo much.
@statquest
Жыл бұрын
Thank you!
@Luxcium
7 ай бұрын
I am someone who loves *SQ,* and *JS* style of teaching in byte 😅 pieces but I also hate _snakes…_ I love *JavaScript* and *TypeScript* but I’ve been learning *JavaScript* with the _strictest linting rules_ one would imagine… and given how *JavaScript* could be used without any sort of strict rules (and is very similar to *Python* in this context) it is frustrating that it makes *Python* very hard to understand despite being easier since it has not the same stricter rules I have imposed myself learning *JavaScript…* but I am also genuinely grateful that *JS* is the best instructor for this kind of topics because *JS* has a _Ukulele,_ *StatSquatch* and *Normalsaurus* which are all there to help *JS* make *SQ* awesome 🎉🎉🎉🎉 Thanks 😅😅😅❤
@statquest
7 ай бұрын
bam!
@Ajeet-Yadav-IIITD
2 жыл бұрын
Thank you Josh, pls continue this series of pytorch!
@statquest
2 жыл бұрын
Will do!
@aayushjariwala6256
2 жыл бұрын
It amazes me, when I see no NLP video on StatQuest! Josh your explanation are always higher than what one can expect and you have created so many series including maths and conceptual understanding. NLP has the same importance compared to computer vision and actually people are suffering to learn it by lack of content availability! I hope you would create a series or maybe a few videos on basic concepts which help people to get interested in NLP : ) Hope you are doing good in life Josh
@statquest
2 жыл бұрын
I'm working on NLP.
@vans4lyf2013
2 жыл бұрын
@@statquest Yay so glad to hear this, we really need you because no one gives great explanations like you do. Also your youtube comments are the nicest I've ever seen which is a testament to how valued you are in this community.
@statquest
2 жыл бұрын
@@vans4lyf2013 Thank you very much!
@jessicas2978
2 жыл бұрын
Thank you so much, Josh. I have been learning PyTorch and deep learning. This video helps me a lot!
@statquest
2 жыл бұрын
Great to hear!
@arijitchaki1884
8 ай бұрын
Hi Josh, sorry to be a spoil sport, but I used exact same code and my prediction is showing 0.5 for dosage of 0.5 and it is running for all 100 epoch and final b value comes out to be -16.51 😔. But yes the concept is clear!! Great work! I always ask people whoever are interested in learning about data science or machine learning to refer you channel. Seeing your channel grow from 10-20K to a Mn is pleasure to my eyes!! You are the "El Professor"!!
@statquest
8 ай бұрын
Thank you very much! If you look at my actual code (follow the link), you'll see that I actually pulled a trick with the data to get it to train faster.
@veronikaberezhnaia248
2 жыл бұрын
Amazing content, as always. Before I was a bit afraid to start closing in torch, so thank you to encourage le to do that!
@statquest
2 жыл бұрын
bam! You can do it! :)
@mahammadodj
2 жыл бұрын
Thank you very much! I am new to Deep Learning. I can say that just in one week i learned a lot of things from your tutorials!
@statquest
2 жыл бұрын
Happy to hear that!
@ais3153
2 жыл бұрын
BIG LIKE before watching 👍🏻 please continue the pytorch series
@statquest
2 жыл бұрын
Will do! :)
@saralagrawal7449
4 ай бұрын
Just watched matrix multiplication of Transformers. My mind was blown away. Same things appear so complex but when this guy explains them, it's like peanuts. Triple BAM
@statquest
4 ай бұрын
Bam! :)
@aabshaarahmad7853
2 жыл бұрын
Hi! This is amazing. Are you gonna continue this series? Out of ten different rabbitholes I have been to, this video has been the most helpful for me with understanding PyTorch and starting off with my project. Please continue making more complicated models. Thank you :)
@statquest
2 жыл бұрын
That's the plan!
@nicolasreinaldet732
2 жыл бұрын
Guess who was going to start programing a neural network in python today...... God bless you Josh, becase He know how much you are blessing me with your work. And know that Jesus loves you and want to be part of your life.
@statquest
2 жыл бұрын
Thanks!
@_epe2590
2 жыл бұрын
finally! some simple to understand content on how to make an AI model using pytourch!!! TRIPLE BAM!!!!
@statquest
2 жыл бұрын
Hooray!
@the_real_cookiez
2 жыл бұрын
Quality educational content! It's so cool to see your channel grow. Been here since ~90k subs! Very well earned.
@statquest
2 жыл бұрын
Wow! Thank you very much!!! BAM! :)
@shamshersingh9680
5 ай бұрын
Hi Josh, thanks again for allowing me to break the ice between me and Pytorch. Everytime I see your videos, I wonder if my instructor could have taught us like this probably our lives must have been much simpler and happier. I have a small doubt here. In the example you have shown gradient training of only final bias. But in reality, all the weights will have to be trained during backpropagation. So when I try to initialise the all weights with random values and then train the model, I do not get the final weights as shown in the video. The code is as follows :- class BasicNN(nn.Module): def __init__(self): super().__init__() self.w00 = nn.Parameter(torch.randn(1), requires_grad = True) self.b00 = nn.Parameter(torch.randn(1), requires_grad = True) self.w01 = nn.Parameter(torch.randn(1), requires_grad = True) self.w10 = nn.Parameter(torch.randn(1), requires_grad = True) self.b10 = nn.Parameter(torch.randn(1), requires_grad = True) self.w11 = nn.Parameter(torch.randn(1), requires_grad = True) self.b_final = nn.Parameter(torch.randn(1), requires_grad = True) def forward(self, input): input_top_relu = input * self.w00 + self.b00 input_bottom_relu = input * self.w10 + self.b10 output_top_relu = F.relu(input_top_relu) * self.w01 output_bottom_relu = F.relu(input_bottom_relu) * self.w11 input_final_relu = output_top_relu + output_bottom_relu + self.b_final output = F.relu(input_final_relu) return output # Create an instance of the neural network model = BasicNN() # Print parameters print('Parameters before training') for name, param in model.named_parameters(): print(name, param.data) # Define inputs and corresponding labels inputs = torch.tensor([0., 0.5, 0.1]) labels = torch.tensor([0., 1.0, 0.]) # Define a loss function criterion = nn.MSELoss() # Define an optimizer optimizer = optim.SGD(model.parameters(), lr=0.01) # Number of epochs for training epochs = 1000 # Training loop for epoch in range(epochs): total_loss = 0 # Forward pass output = model(inputs) # Compute the loss loss = criterion(output, labels) total_loss += loss # Backward pass loss.backward() # Compute gradients optimizer.step() # Update weights optimizer.zero_grad() # Clear previous gradients # Print loss every 100 epochs if (epoch + 1) % 100 == 0: print(f"Epoch [{epoch+1}/{epochs}], Loss: {loss.item()}") if (total_loss < 0.00001): print(f'Epoch = {epoch}') break # Print final parameters print('Parameters after training') for name, param in model.named_parameters(): print(name, param.data) # check the model performance input_doses = torch.linspace(start = 0, end = 1, steps = 11) output = model(input_doses) sns.set(style = 'whitegrid') sns.lineplot(x = input_doses, y = output.detach(), color = 'green', linewidth = 2) plt.xlabel("Input Doses") plt.ylabel("Effectiveness") plt.show() Request if you can help me with the code above.
@statquest
5 ай бұрын
This example only works to optimize the final bias term.
@amirhosseinafkhami2606
2 жыл бұрын
Great explanation as always! Thanks for making content like this, which complements the theoretical concepts.
@statquest
2 жыл бұрын
Glad you liked it!
@xedvap
2 жыл бұрын
Looking forward to seeing your following videos! Excellent explanation!
@statquest
2 жыл бұрын
Awesome, thank you!
@exxzxxe
Жыл бұрын
Another charming, fully informative masterpiece.
@statquest
Жыл бұрын
Thank you very much! BAM! :)
@praptithapaliya6570
Жыл бұрын
I love you Josh. God bless you. You're my favorite teacher.
@statquest
Жыл бұрын
Thank you! 😃!
@justinhuang8034
2 жыл бұрын
Man the content keeps getting better
@statquest
2 жыл бұрын
Thank you!
@Irrazzo
Жыл бұрын
Thank you, good explanation! 16:00 Python prefers for-each-loops over index-based loops. See how this equivalent for-each loop looks much simpler. for input, label in zip(inputs, labels): output = model(input) loss = (output - label)**2 loss.backward() total_loss += float(loss)
@statquest
Жыл бұрын
Great tip!
@WeeeAffandi
2 жыл бұрын
Josh explaining the code is far better than any programmer
@statquest
2 жыл бұрын
Thank you!
@ShawnShi-hy9ed
4 ай бұрын
Hi Josh, thanks for your video. I am confused why it doesn't work when I am trying to optimize any other weights and bias. five minutes later, I think I have got the answer from the comments and your reply. Thanks again!
@statquest
4 ай бұрын
bam
@kleanthivoutsadaki5989
Жыл бұрын
thanks Josh, you really make understanding Neural Networks concepts a great process!
@statquest
Жыл бұрын
Thank you! :)
@ISK_VAGR
2 жыл бұрын
That is a big leap. I need to check it several times to understand it since I am not a programmer. However, I really got a good feeling of what is happening inside the code. I actually use codeless systems such as KNIME. So if Mr. Sasquatch, get the idea of using KNIME to explain all this, It will be amazing. Thanks to be such a good teacher.
@statquest
2 жыл бұрын
I'll keep that in mind.
@MariaHendrikx
11 ай бұрын
I love how you you visualize and synchronize the code with the maths behind it :) On top of that you are doing it step-wise which results in a really awesome and very eduSupercalifragilisticexpialidociouscational video! #ThankYou
@statquest
11 ай бұрын
I love it. Thank you very much! :)
@sceaserjulius9476
2 жыл бұрын
I am also learning Deep Learning, and want to apply it to make good projects, This is going to be great.
@statquest
2 жыл бұрын
bam!
@joshstat8114
7 ай бұрын
Nice video for the introduction of LSTM using PyTorch. There is also `torch` R package that doesn't need to install python and torch. It's so nice that R also has deep learning framework aside from `tensorflow` and I recommend you to maybe try it.
@statquest
7 ай бұрын
Thanks for the info!
@joshstat8114
7 ай бұрын
@@statquest i strongly recommend it because it is so nice that R has own deep learning frameworks, besides h2o
@theblueplanet3576
7 ай бұрын
Enjoying this series on machine learning. By the way there is no shame in self promotion, you deserve it 😁
@statquest
7 ай бұрын
Thanks 😅
@kwang-jebaeg2460
2 жыл бұрын
Wonderful !!! Cant wait your pytorch lightning code for NN. Always thanks alot !!
@statquest
2 жыл бұрын
bam! :)
@Luxcium
Жыл бұрын
Wow 😮 I didn't knew I had to watch the *Neural Networks part 2* before I can watch the *The StatQuest Introduction To PyTorch* before I can watch the *Introduction to coding neural networks with PyTorch and Lightning* 🌩️ (it’s something related to the cloud I understand) I am genuinely so happy to learn about that stuff with you Josh❤ I will go watch the other videos first and then I will back propagate to this video...
@statquest
Жыл бұрын
Warmer...
@Hitesh10able
Жыл бұрын
Another excellent video, one humble request please provide video on Stable Diffusion Models.
@statquest
Жыл бұрын
I'll keep that in mind.
@sabaaslam781
Жыл бұрын
Hi Josh. I am a big fan of your videos. I have a question regarding this quest. In this video, we optimized only one parameter. How can we optimize all the parameters? Thanks in advance.
@statquest
Жыл бұрын
I show how to impute all of the parameters in this video on LSTMs in PyTorch: kzitem.info/news/bejne/s359z4yGrqGQo34 (if you want to learn about the theory of LSTMs, see: kzitem.info/news/bejne/unmwsm1sp35onWU
@emilyli6763
2 жыл бұрын
honestly wish I had this a year ago when I was struggling, still watching now tho!
@statquest
2 жыл бұрын
Bam! :)
@StratosFair
2 жыл бұрын
Nice video, looking forward to the next ones on Pytorch Lightning !
@statquest
2 жыл бұрын
Me too! BAM! :)
@AHMAD9087
2 жыл бұрын
The tutorial we all needed 🙂
@statquest
2 жыл бұрын
Hooray!
@neginamirabadi4595
2 жыл бұрын
Hello Josh! Thank you so much for your amazing videos! I have learned so much from your tutorials and would not have been able to advance without them! I wanted to ask whether it is possible for you to put some videos on times series analysis, including autoregression (AR), moving average (MA) and their combinations. I would be more than grateful if you can provide such a video. Thank you so much.
@statquest
2 жыл бұрын
I'll keep those topics in mind!
@stivraptor
2 жыл бұрын
Hey Josh! Guess what just arrived in the mail.... My new statquest mug!!!!! Hooray!!!
@statquest
2 жыл бұрын
BAM!!! Thank you so much for supporting StatQuest!!!
@mohsenmoghimbegloo
2 жыл бұрын
Thank you very much Mr Josh Starmer
@statquest
2 жыл бұрын
Thanks!
@someone5781
2 жыл бұрын
Woo! Been waiting for this sort of a tutorial!!!
@statquest
2 жыл бұрын
bam!
@05747100
2 жыл бұрын
Thanks a lot, beg for Pytorch Series playlist.
@statquest
2 жыл бұрын
Soon!
@darshuetube
2 жыл бұрын
it's great that you are making videos on coding as well.
@statquest
2 жыл бұрын
Thank you!
@Sandeepkumar-dm2bp
2 жыл бұрын
very well explained, thank you for providing quality content, it's very helpful
@statquest
2 жыл бұрын
Glad it was helpful!
@carlitosvh91
5 ай бұрын
Great explanation. Thank you very much
@statquest
5 ай бұрын
Thanks!
@peterkanini867
Жыл бұрын
Please make an entire tutorial about the ins and outs of PyTorch!
@statquest
Жыл бұрын
I've made several PyTorch videos and will continue to make more. You can find the others here: statquest.org/video-index/
@yashsurange7648
2 жыл бұрын
Thanks for this amazing walk through.
@statquest
2 жыл бұрын
Thanks!
@isaacpeng3625
2 жыл бұрын
great video and explanation! me have been struggling in pytorch coding
@statquest
2 жыл бұрын
Bam! :)
@xray788
10 күн бұрын
Hi Josh, I've watched most of your playlist. It is amazing how you explain it. But can you please explain or point to some reference on where the values for weights come from? I see at start of video like w is 1.70 but confuses me where it came from and why those values are used. Thank you Josh and hopefully once i get that it will be a... Triple bam for me :)
@statquest
9 күн бұрын
To create this network, I gave each weight and bias a random initialization value and then tried to fit the neural network to the training data with backpropagation. I then repeated the process a ton of times until I discovered a set of initialization values that worked.
@vtphan2012
2 жыл бұрын
Hi Josh, did you look at and consider Keras before making this video? What do you think about it versus Pytorch?
@statquest
2 жыл бұрын
To be honest, I've only ever worked with PyTorch.
@sagartalagatti1594
2 ай бұрын
Amazing... Can you please tell me how to optimize all the parameters starting with random initial values like we did in "Going Bonkers with Chain Rule"?? I tried some modifications on my own, but couldn't get the result. Help would be greatly appreciated.
@statquest
2 ай бұрын
Unfortunately this model is not a good one for that. Instead, try this: kzitem.info/news/bejne/spxmnIx6kop0i34 and github.com/StatQuest/word_embedding_with_pytorch_and_lightning
@Thamizhadi
2 жыл бұрын
Hi Josh, thank you for introducing pytorch to me. I have an off topic question. How do you create your videos? They look like a series of animated slides. I want to emulate your style for creating presentation slides.
@statquest
2 жыл бұрын
I give away all of my secrets in this video: kzitem.info/news/bejne/xKiCvn59Zndym6A
@jwilliams8210
Жыл бұрын
Absolutely brilliant!
@statquest
Жыл бұрын
Thank you! :)
@junqichen6241
2 жыл бұрын
Hi Josh, could you explain how this line of code works? output_values = model(input_doses). My understanding is model has a method called forward, so shouldn't it be output_values = model.forward(input_doses)?
@statquest
2 жыл бұрын
I answer that question at 9:13
@karrde666666
2 жыл бұрын
better than MIT or any university slides
@statquest
2 жыл бұрын
Thank you!
@random-hj1gv
7 ай бұрын
Hello! I've got a bit confused: at 15:08 you mention that at each epoch we'll be running all 3 data points through the model, but wasn't the point of SDG in that we would only need a single data point per epoch, or am I misunderstanding something? Btw, despite my confusion, this is by far the best ML guide series I've seen, thank you for your work!
@statquest
7 ай бұрын
That's a good question. "torch.optim" doesn't have a gradient descent optimizer, just a stochastic gradient descent optimizer. So we import torch.optim.SGD and then pass it all of the residuals to get gradient descent.
@random-hj1gv
7 ай бұрын
@@statquest Makes sense, thank you for the clarification!
@silveromayo4148
Жыл бұрын
Thanks for the great video. Does this apply directly to GNN? Can I apply it there?
@statquest
Жыл бұрын
To be honest, I don't know much about GNNs right now so I can't answer your question.
@kobic8
Жыл бұрын
great presentation!! thanks again for simplfying this topic! are you planning to post more on NN implementation? computer vision maybe or object detection?
@statquest
Жыл бұрын
Yes, there will be many more videos on how to implement NNs.
@abbddos
2 жыл бұрын
This was great... I hope you can simplify Tensorflow the same way... big big thank you.
@statquest
2 жыл бұрын
Thanks!
@mohammadidreesbhat1109
Ай бұрын
That is how teaching should be.. Triple Bam
@statquest
Ай бұрын
Thanks!
@anggipermanaharianja6122
2 жыл бұрын
Awesome vid by the legend!
@statquest
2 жыл бұрын
Thank you!
@pfever
2 жыл бұрын
Best tutorial like usual! would be nice to see more advanced examples of in pytorch, like CNN for image classification :)
@statquest
2 жыл бұрын
I'm working on them.
@ぶらえんぴん
2 жыл бұрын
Your teaching video is awesome
@statquest
2 жыл бұрын
Thank you!
@ぶらえんぴん
2 жыл бұрын
@@statquest Do you have intro to lightning ? I kind of remember you mentioned in the video you seemed to have one?
@statquest
2 жыл бұрын
@@ぶらえんぴん That's going to be the next video in this series. It will come out in a few weeks.
@arer90
2 жыл бұрын
Thank you for perfect lecture~!!!
@statquest
2 жыл бұрын
Thank you!
@AIpodcast369
2 жыл бұрын
Sir, Please make videos on the time-series analysis, it's hard to find the videos with clear explaination.
@statquest
2 жыл бұрын
I'll keep that in mind.
@pauledam2174
Жыл бұрын
Amazing job! I plan to donate to your patreon page. You were confused because we could use .backward on loss (or at least I was confused by this). I guess one explanation is that loss is defined in terms of output_i and output_i is an instance of the model class. So it may make sense that we can access the backward attribute of loss. But I was, for the same reason a bit surprised that we can subtract a scalar from output_i. One other question. Wouldn't it be better to take the average of total loss? Otherwise the condition that uses 0.0001 is dependent on the the number of examples in the training set.
@statquest
Жыл бұрын
Taking the average is really common, but it doesn't change anything.
@pauledam2174
Жыл бұрын
No, I know it doesn't change things but it means that your criteria has to be adjusted depending on the size of the dataset doesn't it?
@statquest
Жыл бұрын
@@pauledam2174 If we wanted to compare the loss among different datasets, then the average would be helpful.
@conlele350
2 жыл бұрын
Thanks Josh, its Incredible video. Beside, recently the Bayes theorem application in fitting model (linear, logistic, random forest...) has became more and more popular in order to replace classic statistic method, could you pls take some time to explain to us some of its popular algorithm like BART, Linear regression via Bayesian Methods...
@statquest
2 жыл бұрын
I'm planning on doing a whole series on Bayesian stuff as soon as I finish this series on neural networks.
@conlele350
2 жыл бұрын
@@statquest that's great news for today, thanks Josh, Im looking forward to see it soon
@milanvarady
2 жыл бұрын
If I would like the neural network to optimize all the other parameters by itself and not just the final bias, how would I go about that? Or is it even possible with such a small network? I tried setting the other parameters to 0 and requires_grad to True, but that doesn't seem to work.
@statquest
2 жыл бұрын
To be honest, I'm not certain why it is so hard to train all of the parameters at the same time in this neural network. It seems like there are tons of local minimums, and unless you get really lucky with the initial values for each parameter, you will get stuck in a local minimum and fail to get to the global minimum. This may be a function of the simplicity of the neural network - I built this one by hand by simplifying a more complex neural network. The simplification was needed so that I could easily draw it on the screen.
@jawadmansoor6064
2 жыл бұрын
Great series.
@statquest
2 жыл бұрын
Thank you!
@vighneshsrinivasabalaji802
11 ай бұрын
Hey Josh! Amazing videos, thanks a lot. Would be great if you could cover Time Series Data and algorithms like ARIMA and HOLTS WINTER Thanks😊
@statquest
11 ай бұрын
I'll keep those topics in mind.
@Luxcium
7 ай бұрын
So I paused at 18:04 because _it blew my mind_ that we were calling backwards() on the loss variable because I thought it was defined on the line above… 😅😅😅😅 but yeah I didn’t find anything so one hour later I was just watching the rest of the video and _to be honest_ in about 33 seconds it came out that it was normal for _my mind to be blown_ 😂😂😂😂 at 18:37
@statquest
7 ай бұрын
Totally! I was like, "what?!?!?" when I first saw that.
@Emily-Bo
2 жыл бұрын
Awesome video! Thanks, Josh! Can you please explain what super() does in the _init_()?
@statquest
2 жыл бұрын
Great question! So, we're making a new class that is derived from nn.Module, and nn.Module, is derived from something else, and all those things need to be initialized, so "super()" does that for us.
@guramikeretchashvili1569
2 жыл бұрын
So interesting videos and good explanations. I am wondering which software you use to make these cool visualizations?
@statquest
2 жыл бұрын
I share all my secrets here: kzitem.info/news/bejne/xKiCvn59Zndym6A
@lloydchan9606
2 жыл бұрын
bless josh and this channel
@statquest
2 жыл бұрын
Thank you!
@HtHt-in7vt
2 жыл бұрын
I would be appreciated if you can teach more an deeper in pytorch. Thank you so much!
@statquest
2 жыл бұрын
That's the plan. This is just the first of many videos on how to code neural networks. The next video will be on pytorch lightning, and then we'll start to create more advanced models.
@gabrielgirodo
Жыл бұрын
Thank you, Josh for all the help you have been proving to me, you have no idea! I was curious on how to optimize all biases and weights just like we saw on the Quest however, the "optimizing function" seems to only be able to optimize the final_bias variable, I could not understand why. Is it something connected to the pytorch library and the optimizer variable? Have a great week!
@statquest
Жыл бұрын
The problem is that we have very little data. I actually had to fit this neural network by hand to make it work to begin with. I did this because I wanted a very simple NN to demonstrate the principles.
@gabrielgirodo
Жыл бұрын
@@statquest Thank you for the reply! :)
@frankyang442
Жыл бұрын
Thanks for teaching in such a simple way ! But I'm a little bit confused about the code below: total_loss += float(loss) since you accumulate the loss ( the implementation is inside the model) will this step double-counting the loss ? (or it is just like prefix sum ... )
@frankyang442
Жыл бұрын
And btw will you upload the transformer in near future or the code to update all the parameter so that everybody can understand the whole concept of back propagation ?
@statquest
Жыл бұрын
I'm not sure I understand your question. Where else is the loss accumulated?
@statquest
Жыл бұрын
1) My video on Transformers will come out soon. 2a) If you want to understand the concept of backpropagation, I suggest: kzitem.info/news/bejne/qoRovqF4oXt9p2k kzitem.info/news/bejne/yq-kmK6aen6anG0 and kzitem.info/news/bejne/qIGQ1YN-kXmUf6Q 2b) If, instead, you'd just like to use PyTorch to optimize all of the parameters in a model, see kzitem.info/news/bejne/s359z4yGrqGQo34
@firuzerehimova6355
10 ай бұрын
one question about 21.07 : i get what optimizer zero grad does, but i though total_loss =0 does the same thing at the start of nested loop? also for some reason my final bias ended up at -709 after 100 epochs, maybe thereis a problem in the way i was writing the code (and couldnt fill out the form to download from your website, maybe you should check that) but i am grateful to you for making this video i think i learned a lot!!
@statquest
10 ай бұрын
Thanks! You can download the code here: github.com/StatQuest/pytorch_lightning_tutorials "total_loss" keeps track of the loss, the difference between the observed and the predicted values. The optimizer, however, keeps track of the gradients (the derivatives with respect to each variable), and those need to get zeroed out.
@sergiochavezlazo5362
Жыл бұрын
Hello! Thank u so much for this video. What is the difference between Keras and Pytorch?
@statquest
Жыл бұрын
I've never used Keras, but I believe it's a simple framework for solving specific problems and lacks flexibility to do other problems.
@konstantinostsoumas6390
2 жыл бұрын
This is wonderful content! Do you have any news regarding the book perhaps? I think it was about to drop at end of April, am I correct?
@statquest
2 жыл бұрын
The book is done! However, I still have a few details to work out on how exactly to publish it.
@konstantinostsoumas6390
2 жыл бұрын
@@statquest wonderful news! Do you believe we can purchase it from May onwards then?
@statquest
2 жыл бұрын
@@konstantinostsoumas6390 I hope to have at least the digital version available in may (no exact date yet - it depends on a few meetings I'm supposed to have this week).
@bjornnorenjobb
2 жыл бұрын
omg! I have really wanted this! awesome!!! :) :) :)
Пікірлер: 357