My evolution

  • Started with learning physics.
  • Continued with educational physics, with the purpose to build a computer software to serve as a tool to provide a more comprehensible and more understandable physics. It is called SciSoft.
    Partcicpated in Prof. Miriam Reiner’s NeuroCognition in Education Lab.
    Though the tool was not expressed practically, its theoretical framework can be used later in AGI.
  • Then continued with totally different field: Optimal control applied in transportation.
    Partcipated in Prof. Jack Haddad’s TSMART Lab.
  • Then moved on into AI field, applying Deep Learning (Supervised Learning via Graph Neural Network and Reinforcement Learning via Deep Q-Learning) in signal control problem at transportation network.
  • During my studying the AI field, then Machine Learning and then Deep Learning, I encountered the fabulous and fascinating world of Artificial General Intelligence (AGI).

Overview of gained knowledge

Note: All formal courses were taken in Technion Institute in Israel.

Control Theory (2017-2019):

Courses Title Main Ideas
Formal Optimal control 016713 Pontryagin Maximum Principle (PMP), Lagrange multipliers, Sliding mode, Linear control,
Linear Quadratic Regulator (LQR), Continous Knapsack Problem (CKP), Krotov Lemma
Non-Linear control 086312 1st-, 2nd- and High-Order Systems, Phase Diagram, State Space, Region Of Attraction, Systems: (Non-)Autonoums, Perturbed, Open/Closed-Loop, with input/output - and their Lyapunov stability conditions. BIBO, ISS, Canonical Forms: Normal and Controller - and their Control Design. Causality, Dynamic Systems, Tracking, Sliding Mode Control (SMC)
Self learning System Analysis 014004 Convexity, Linear Programming (LP), Simplex, KKT conditions, Primal/Dual Problem,
(Mixed) Integer LP, Dynamic programming (DP), Decision under uncertainty
Linear Systems 034032 Discrete/Continuous Systems, Transformation: Laplace, Z, Fourier. Time/Frequency domain. Canonical Forms: Observer, Controller. Stationarity, Convolution, Different-order Systems, Feedback Control, LTI, SISO, Stability analysis: poles and zeros, Transient and Steady-state. Bode Plot, Routh–Hurwitz stability criterion, Asymptotic stabilities, Impulse response
Introduction to Control 034040 Block diagram, Open/Closed-loop control, Plant Inversion, Internal Stability,
Root-Locus method, Nyquist stability criterion, Delay/Dead-time, Gain, Lead/lag Compensator,
Basic loop shaping and stability margins, P/PI/PID controllers
Control Theory 035188 MIMO, Advanced loop shaping, Pole placement, Nychols Chart, Kalman Filtering, LQG,
Sampled-data control: connecting analog and digital, control of dicrete-time systems, aliasing. Controllability, Observability, Estimators, Robustness, Transfer functions
Adaptive Control (books) Signal/System Norm, Lyapunov stability: Invariant sets theorem, Time Varying System Stability, Barablat’s Lemma. Direct control: model reference adaptive control (MRAC), Indirect control: self-tuning-regulator (STR). Linearization, Sliding Variables, Robust Control, Dynamic Inversion
Systems and Control 017003 Quadratic Programming (QP), Multiparametric Programming, Model Predictive Control (MPC), Receding Horizon Control (RHC), Vertex Control, Set theory: Ellipsoidal/Polyhedral set, Nominal/Robust State/Output feedback Interpolating control (IC), Interpolating with elliptic-sets/cost, Constrained optimal control, Linear Matrix Inequality (LMI), Implicit/Explicit solution

Other courses completed in Technion:

  1. Statistics (Linear Regression, Random variables, Hypothesis Testing, Sampling, Estimation, Monte-Carlo models, ANOVA) 019007
  2. Demand Modelling (Discrete Choice Methods, Maximum Likelihod Estimation (MLE), Hypothesis Testing, Logit, Probit, Utility) 019710
  3. Network theory (Maximum flow problem, Shortest-Path algorithms, Minimum spanning tree) 019006
  4. Transportation Network Analysis (Equilibrium Problem, Optimal routing, Route Choice Models) 019709
  5. Traffic control 019718
  6. Transportation Engineering and Management 014732
  7. Transportation Planning 014702
  8. Advanced Transportation Engineering (Microscopic/mesoscopic/macroscopic traffic models, Macroscopic Fundamental Diagram) 019714

Artificial Intelligence (2019-today):

Type Title Main Ideas Resources Internal ref
Formal Courses Introduction to Machine Learning PAC learning, VC dimension/theory, Empirical risk minimization (ERM), Bias-Variance tradeoff, No-free lunch, Non-uniform learnability, MDL, Boosting, SVM (support vector machine), Kernels, regularization, online learning, Feature selection, multilayer networks,
Probabilistic models, Naive Bayes, generative vs discriminative models, MAP (maximum a posteriori) vs SRM (structural risk minimization), Linear Discriminant Analysis (LDA), Expectaion-Maximization (EM), PCA, Multi-class classification		 236756 AICourses.docx
Deep Learning PAC Bayes, dropout, KL/JS divergence, Batch Normalization (BN), Adaptive learning,
CNNs (ResNet, GoogleNet, ..),
RNNs (LSTM, GRU),
Transformers (attention, GNN, multi-modal, VQA),
Generative learning ((Style/Cycle) GAN, discrete/continuous VAE, Gradient Langevin dynamics),
Reinforcement Learning (value-based, policy-gradient, Actor-Critic (AC), A2C, A3C),
Explainability (LIME, SHAP, Gradient-based)		 097200 DL.docx
Introduction to Natural Language Processing Set/Bag of words, Distance metrics (Hamming/Jaccard/Euclidean/Cosine), Classification (Majority, Neirest Neighbor, Naïve-Bayes), Text normalization (Tokenization, Lemmatization, stemming), Regular expression (RE), true/false positive/negative, POR plot, Cross-validation, bootstrap, Laplace smoothing, N-gram/Neural Language model (LM), perplexity, Hidden Markov model (HMM), Word Embeddings (word2vec), POS tagging, named entity recognition (NER), Viterbi algorithm,
(Probabilistic/Weighted) context-free grammar (CFG), Chomsky normal form, Semantic Parsing, lambda calculus, Seq2Seq (encoder-decoder, attention), Quantifiers, Contextual Embeddings (BERT, BART), beam search, machine translation (MT)		 236299 Intro_to_NLP.docx
Self learning Courses Deep Learning in Computer Vision different CNN architectures, Region-based CNNs (R-CNNs, SSD, YOLO) Coursera CNN_images.docx
The Ancient Secrets of Computer Vision different CNN architectures, Region-based CNNs (R-CNNs, SSD, YOLO) Course
Machine Learning courses percepton learning algorithm, linear classification/regression, logirstic regression, multilayer percepton, SVM, Kernel methods, Radial Basis Function (RBF), learning principles: Occam’s razor, Sampling bias, Data snooping; Bayesian methods, Aggregation methods. Random Forest, Clustering and Mixture Models e.g. GMM, Probabilistic Graphical Models, (Hidden) Markov Models, Undirected Graphical Models, MRF, Belief Propagation, Adversarial Search, Markov decision processes (MDP), Logic 1,2,3,4 AICourses.docx
Deep Learning courses Deep belief networks (DBN), Restricted Boltzmann networks (RBM), BM, Auto-Encoder, Sparse coding, vanishing/exploding gradient, KL divergence, and more 1,2,3,4
Connectionism in GOFAI or classical AI (20th century) Basic learning rules, e.g. Hebbian, Memory-based, Competitive, Boltzmann. Associative memory, RBF (radial basis function), MLP (multi-layered perceptron),
SOM (self-organizing-map)		 Youtube
Multi-task and Meta learning Model Agnostic Meta Learning (MAML), Meta-Learning Approaches: Black-box, Optimization-based, Non-parametric. Zero/Few-shot learning, Bayesian meta-learning, Meta-RL, Domain generalization/adaptation, Lifelong learning. Contrastive Learning Site, Youtube, Another
Causality A/B testing, Graphical models, Causal Influence Diagrams, Counterfactuals
Models of computation Finite state machines (via Regular language), Pushdown automaton (via Context-free language), Turing machines (via Recursively enumerable language), Recursive language, Decidable Languages, Grammars, Chomsky hierarchy Campus
Introduction to (Classical) AI Top-down (Symbolic AI) verse Bottom-Up (Neural Networks) approaches; Knowledge Representations (OAV tripplets, Semantic Network, Conceptual Graphs, Production Rules, Frames and Scripts/Scenarios, and Logic: descriptive, propositional, predicate/1st order, 2nd order, high order); Expert Systems, Knowledge Base, and Inference/Rule Engine; Cognitive architecture/system; Generative-test and Means-Ends Analysis; Case-based-, Commonsense-, Analogical-, and Meta- Reasoning; Learning: Recording cases, Explanation-based, and Incremental Concept; Configuration and Diagnosis; Constraint Propagation, Version Spaces, Planning and Understanding;
de/in/ab-duction		 1, 2, 3, 4
Abstract Algebra and Category theory Group, Set, Category, [more in process] 1,2
Books and Articles Copeland, Jack Artificial Intelligence: A Philosophical Introduction (1993) Book AIsummary.docx
Jean-Claude Bringuier Conversations with Jean Piaget (1977) Book
Edward de Bono Parallel/Lateral Thinking (1995) Book
I Am Right, You Are Wrong (1990) Book
French, C. C., & Colman, A. M. Cognitive psychology (1995) Book
Elizabeth F. Loftus Visual Perception: An Introduction (2012) Book
Jeff Hawkins On Intelligence (2004) Book
A Thousand Brains: A New Theory of Intelligence (2021) Book
Elaine Rich, Kevin Knight Artificial Intelligence (1991) Book