Introduction and Perceptron
-
01010_Intro (6:24)
-
01020_Why_Write_a_Neural_Network (4:57)
-
01025_Getting_the_Most_Out_of_This_Course (5:16)
-
01030_Java_vs_Python (9:32)
-
01040_Neurons (3:40)
-
01050_Perceptron (5:20)
-
01060_Project_With_JUnit_Support (6:18)
-
01070_Coding_Perceptron (6:34)
-
01080_Where_to_Find_the_Source_Code (3:01)
-
01090_Eclipse_Formatters (1:19)
-
01100_Logic_Gates (5:56)
-
01110_Perceptron_AND (9:37)
-
01120_OR_NOR_NAND (6:40)
-
01130_XOR_and_XNOR (5:04)
-
01140_Linear_Separability (6:36)
-
01150_Some_Layer_Terminology (5:10)
-
01160_Labelling_Weights (5:49)
-
01170_Matrices (5:07)
-
01180_Some_Mathematical_Terminology (4:08)
Matrix Mathematics
-
02010_A_Matrix_Class (4:30)
-
02020_Initialising_the_Matrix (8:36)
-
02030_Matrix_toString_Method (6:40)
-
02040_Testing_the_toString_Method (9:58)
-
02050_The_Apply_Method (3:58)
-
02060_Multiplying_Matrices_By_a_Value (5:21)
-
02070_Comparing_Matrices (5:07)
-
02080_Using_the_Equals_Method (6:08)
-
02090_Adding_Matrices (5:28)
-
02110_Motivation_for_Matrix_Multiplication (4:19)
-
02120_Multiplying_the_Tables
-
02130_Matrix_Multiplication (6:00)
-
02140_Matrix_Multiplication_Rule (4:47)
-
02150_Matrix_Multiplication_Summary (4:49)
-
02160_Matrix_Multiplication_Examples (8:09)
-
02170_Assertions (5:06)
-
02180_2D_to_1D (5:52)
-
02190_Iterating_Over_Multiplicand_Rows (4:00)
-
02200_Completing_the_Multiplication_Implementation (5:29)
-
02210_Timing_Matrix_Multiplication (6:07)
-
02220_Optimising_Matrix_Multiplication (6:36)
Activation Functions
-
03010_Neural_Net_Test_Class (4:35)
-
03020_Modifying_Matrices (3:12)
-
03030_Adding_Bias (1:45)
-
03040_Multiple_Columns_of_Input (4:59)
-
03050_ReLu (4:36)
-
03060_A_ReLu_Test (5:30)
-
03070_Matrix_forEach (4:22)
-
03080_Implementing_ReLu (4:48)
-
03090_Introducing_Softmax (4:48)
-
03100_Softmax_Worked_Example (2:37)
-
03110_Summing_Columns (4:55)
-
03120_Implementing_Softmax (3:15)
-
03130_Testing_Softmax (6:36)
-
03140_The_Engine (6:22)
-
03150_Deciding_Weight_Matrix_Sizes (5:54)
-
03160_An_Untrained_Network (7:02)
-
03170_Configuring_Dense_Layers (5:21)
-
03180_Adding_Multiple_Layers (5:54)
-
03190_Running_the_Engine (8:00)
Information Theory and Cross Entropy
-
04010_Mean_Squares_Loss (6:51)
-
04020_What_is_Information (4:22)
-
04030_Symbol_Spaces (6:30)
-
04040_Entropy (6:07)
-
04050_An_Optimal_Encoding_Strategy (6:26)
-
04060_Unequally_Probable_Symbols (4:30)
-
04070_Calculating_the_Information_Assoicated_With_a_Symbol (4:49)
-
04080_Entropy_for_Unequally_Probable_Symbols (5:45)
-
04090_Introducing_Cross_Entropy (5:28)
-
04100_Cross_Entropy_Example (7:36)
-
04110_Cross_Entropy_as_a_Loss_Function (6:38)
-
04120_Implementing_Cross_Entropy (7:08)
-
04130_A_Cross_Entropy_Test (6:51)
-
04140_Implementing_the_Cross_Entropy_Test (5:45)
Calculus and Backpropagation
-
05010_Training_the_Network (3:36)
-
05020_Gradient_Descent (4:50)
-
05030_Gradients_and_Neural_Networks (4:15)
-
05040_A_Calculus_Class (5:54)
-
05050_Implementing_Differentiation (6:50)
-
05060_Basic_Mathematical_Notation (5:45)
-
05070_Partial_Derivatives (5:06)
-
05080_Overview_of_a_3_Layer_Network (6:29)
-
05090_The_Network_as_a_Transform (5:18)
-
05100_Approximator (5:38)
-
05110_Mock_Expected_Data (5:40)
-
05120_Implementing_the_Transform (5:37)
-
05130_Examining_Loss (5:04)
-
05140_An_AddIncrement_Method (7:01)
-
05150_Completing_the_Approximator (5:02)
-
05160_Recap_and_Natural_Logarithms (6:18)
-
05170_Finishing_the_Approximator_Test (5:30)
-
05180_Back_Propagation (5:31)
-
05190_Obtaining_Softmax_Cross_Entropy_Gradient (7:22)
-
05200_Backpropagating_Errors_Through_Weighted_Sums (3:11)
-
05210_Introducing_the_Chain_Rule (8:04)
-
05220_Programming_the_Chain_Rule (7:21)
-
05230_Chain_Rule_for_Functions_of_Multiple_Variables (7:00)
-
05240_Programming_Multi_Variable_Chain_Rule (8:29)
-
05250_More_Mathematical_Terminology (7:45)
-
05260_Matrix_Transpose (5:30)
-
05270_Implementing_Transpose (5:18)
-
05280_Backpropagation_Through_Weights (4:42)
-
05290_Calculating_the_Backpropagation_Transform_for_Weights (8:40)
-
05300_Creating_a_Test_for_Weights_Backprop (5:22)
-
05310_Creating_Weights_and_Biases (3:53)
-
05320_Completing_the_Weights_Backprop_Test (6:23)
-
05330_Improving_the_Weight_Backprop_Test (3:05)
-
05340_Backpropagating_Through_ReLu (4:44)
-
05350_Implementing_Backpropagation_Through_ReLu (7:41)
The Neural Network Engine
-
06010_Adding_a_RunBackwards_Method (5:58)
-
06020_A_Test_Data_Class (3:04)
-
06030_A_Batch_Result_Class (2:42)
-
06040_Configuring_the_Engine (4:09)
-
06050_Checking_the_Loss_and_Final_Transform (5:41)
-
06060_Storing_Errors (2:44)
-
06070_Initiating_Backpropagation (3:33)
-
06080_Checking_Initial_Backpropagation (5:05)
-
06090_Addiing_Backpropagation_Through_Weights (6:16)
-
06100_Adding_Backpropagation_Through_ReLu (2:49)
-
06110_Turning_Off_Backpropagation_to_Input (2:24)
Training the Neural Network
-
07010_Calculating_the_Weight_Gradients_from_the_Error (9:42)
-
07020_Practical_Weight_Gradient_Calculation_via_Coding (6:37)
-
07030_Completing_the_Weight_Gradients_Test (6:24)
-
07040_A_Useful_Fact_Involving_Matrix_Transpose (5:31)
-
07050_Online_Training_versus_Batch_Training (4:37)
-
07060_Training_the_Network_in_Full (5:23)
-
07070_Testing_Average_Column (4:58)
-
07080_Averaging_Columns (3:21)
-
07090_Evaluating_Average_Loss (5:38)
-
07100_Getting_Weight_Inputs_and_Errors (6:10)
-
07110_Generating_Trainable_Data (7:22)
-
07120_Configuring_the_Engine_Test (3:57)
-
07125_Finishing_the_Adjust_Method (6:21)
-
07130_Getting_Greatest_Row_Numbers (7:08)
-
07140_Adding_Percent_Correct (6:30)
-
07150_Repeated_Training (7:55)
-
07160_Generating_Spherically_Symmetric_Distributions (6:36)
-
07170_Better_Training_Data (9:20)
-
07180_Trying_the_New_Data (6:03)
-
07190_A_Running_Averages_Class (5:01)
-
07200_Using_Running_Averages (6:17)
-
07210_Variable_Learning_Rates (6:53)
-
07220_Testing_Engine_Performance (2:47)
Epochs, Batches and Multithreading
-
08010_The_MetaData_Interface (5:34)
-
08020_The_BatchData_Interface (2:00)
-
08030_The_Loader_Interface (1:52)
-
08040_The_Neural_Network_Class (4:14)
-
08050_Initializing_Matrixes_with_Double_Arrays (3:25)
-
08060_Generating_Training_Arrays (5:24)
-
08070_Some_Abstract_Convenience_Classes (4:40)
-
08080_Creating_a_Test_Loader (7:16)
-
08090_Completing_the_Test_Loader (5:35)
-
08100_Testing_the_Test_Loader (6:30)
-
08105_Summing_Matrix_Elements (1:20)
-
08110_Finishing_the_Loader_Test (6:07)
-
08120_The_Fit_Method (6:59)
-
08130_Running_with_the_Test_Loader (3:18)
-
08140_Running_the_Epochs (2:54)
-
08150_Running_the_Batches (2:30)
-
08160_Implementing_RunBatch (6:11)
-
08170_Adding_Multithreading (5:10)
-
08180_Consuming_the_Batches (7:12)
-
08190_Fixing_the_Execution_Exception (6:20)
-
08200_Outputting_Metrics (6:00)
-
08210_Preventing_NaN (5:07)
-
08220_Improving_toString (6:14)
-
08230_Saving_the_Network (6:14)
-
08240_Adding_a_Load_Method (5:42)
-
08250_Dealing_with_the_Lock (4:19)
-
08260_Adding_a_Predict_Method (5:31)
Loading MNIST Images
-
09010_The_MNIST_Dataset (6:02)
-
09020_Command_Line_Arguments (6:09)
-
09030_Creating_the_Image_Loader (5:14)
-
09040_Opening_the_MNIST_files (6:49)
-
09050_The_MNIST_File_Format (5:17)
-
09060_Reading_Label_Metadata (5:07)
-
09070_Reading_the_Image_MetaData (3:56)
-
09080_Storing_the_MetaData (8:49)
-
09090_Structuring_Batch_Reading (7:56)
-
09100_Reading_Image_Data (7:10)
-
09110_Converting_Bytes_to_Doubles (5:05)
-
09120_Reading_Batches (2:48)
-
09130_Reading_the_Labels (6:04)
-
09135_An_ImageWriter_Class (5:14)
-
09140_Writing_Images (5:16)
-
09150_Determining_Canvas_Width (4:17)
-
09160_Calculating_Pixel_Location (6:10)
-
09170_Writing_the_Montage_Images (7:06)
-
09180_Converting_One_Hot_to_Int (5:44)
-
09190_Writing_the_Labels (6:31)
Putting It All Together
Conclusion
