Fin
Vithttps://www.coursera.org/specializations/machine-learning-reinforcement-finance
Machine Learning and Reinforcement Learning in Finance Специализация
Содержание
1 - Guided Tour of Machine Learning in Finance
- Неделя 1: Artificial Intelligence & Machine Learning
- Неделя 2: Mathematical Foundations of Machine Learning
- Неделя 3: Introduction to Supervised Learning
- Неделя 4: Supervised Learning in Finance
2 - Fundamentals of Machine Learning in Finance
- Неделя 1: Fundamentals of Supervised Learning in Finance
- Неделя 2: Core Concepts of Unsupervised Learning, PCA and Dimensionality Reduction
- Неделя 3: Data Visualization and Clustering
- Неделя 4: Sequence Modeling and Reinforcement Learning
3 - Reinforcement Learning in Finance
- Неделя 1: MDP and Reinforcement Learning
- Неделя 2: MDP model for option pricing: Dynamic Programming Approach
- Неделя 3: MDP model for option pricing - Reinforcement Learning approach
- Неделя 4: RL and INVERSE RL for Portfolio Stock Trading
4 - Overview of Advanced Methods of Reinforcement Learning in Finance
- Неделя 1: Black-Scholes-Merton model, Physics and Reinforcement Learning
- Неделя 2: Reinforcement Learning for Optimal Trading and Market Modeling
- Неделя 3: Perception - Beyond Reinforcement Learning
- Неделя 4: Other Applications of Reinforcement Learning: P-2-P Lending, Cryptocurrency, etc.
1 - Guided Tour of Machine Learning in Finance
Неделя 1: Artificial Intelligence & Machine Learning
- Get a sense of what Machine Learning is and how it relates to Artificial Intelligence.
- Recognize ML in everyday life: algorithms around us.
- Illustrate the need in ML applications.
- Explain core paradigms of ML: Supervised, Unsupervised and Reinforcement Learning.
- Compare ML for Finance with ML in Technology (image and speech recognition, robotics, etc.)
Introduction to the Specialization "Machine Learning and Reinforcement Learning in Finance"
Artificial Intelligence and Machine Learning, Part I
Artificial Intelligence and Machine Learning, Part
Artificial Intelligence and Machine Learning
Machine Learning as a Foundation of Artificial Intelligence, Part I
Machine Learning as a Foundation of Artificial Intelligence, Part II
Machine Learning as a Foundation of Artificial Intelligence, Part III
Machine Learning as a Foundation of Artificial Intelligence
Machine Learning as a Foundation of Artificial Intelligence, Part I
Machine Learning as a Foundation of Artificial Intelligence, Part II
Machine Learning as a Foundation of Artificial Intelligence, Part III
Machine Learning in Finance vs Machine Learning in Tech
Machine Learning in Finance vs Machine Learning in Tech, Part I
Machine Learning in Finance vs Machine Learning in Tech, Part II
Machine Learning in Finance vs Machine Learning in Tech, Part III
Module 1 Assessment
How AI And Automation Will Shape Finance In The Future
https://www.forbes.com/sites/workday/2017/11/03/how-ai-and-automation-will-shape-finance-in-the-future/?sh=325b7bfc481b
Неделя 2: Mathematical Foundations of Machine Learning
- Explain generalization as a goal of machine learning.
- Interpret bias-variance tradeoff in complex and simple models.
- Illustrate on example the No Free Lunch Theorem.
- Interpret a statistical method as the first ML algorithm.
- Describe how different is ML from statistical modeling.
- Choose hyperparameters using cross-validation.
- Apply methods of linear and logistic regressions learned in the course to FDIC banking call report data in the course project.
Generalization and a Bias-Variance Tradeoff
Generalization and a Bias-Variance Tradeoff
The No Free Lunch Theorem
Overfitting and Model Capacity
Linear Regression
Regularization, Validation Set, and Hyper-parameters
Overview of the Supervised Machine Learning in Finance
Module 2 Assessment
Leo Breiman, “Statistical Modeling: The Two Cultures”
Неделя 3: Introduction to Supervised Learning
- Describe sequence learning with Markov models and neural networks
- Describe the concepts and algorithms of HMMs, and present financial use cases.
- Explain how RL provides more practical ways to optimal control problems in finance.
- Solve Absorption Ratio problem via PCA in the course project.
- Discover principles of dynamic programming and the Bellman equation.
Introduction to Neural Networks and Tensor Flow
DataFlow and TensorFlow
A First Demo of TensorFlow
Linear Regression in TensorFlow
Neural Networks
Gradient Descent Optimization
Gradient Descent for Neural Networks
Stochastic Gradient Descent
E. Fama and K. French, “Size and Book-to-Market Factors in Earnings and Returns”, Journal of Finance, vol. 50, no. 1 (1995), pp. 131-155.
J. Piotroski, “Value Investing: The Use of Historical Financial Statement Information to Separate Winners from Losers”, Journal of Accounting Research, Vol. 38, Supplement: Studies on Accounting Information and the Economics of the Firm (2000), pp. 1-41
Module 3 Assessment
Неделя 4: Supervised Learning in Finance
- Develop a user case of application of Supervised Learning algorithms to finance.
- Discuss usages of ML for trading, asset management, banking risk management.
- Apply both Linear and Non-Linear, Neural Network Regression to solve the problem of prediction of Earnings Per Share.
- Use fluently TensorFlow and other packages such as Scikit-Learn and Statsmodels.
Prediction of Earning per Share (EPS) with Scikit-learn and TensorFlow
Regression and Equity Analysis
Fundamental Analysis
Machine Learning with Probabilistic Models (Classification Tasks)
Machine Learning as Model Estimation
Maximum Likelihood Estimation
Probabilistic Classification Models
Logistic Regression for Modeling Bank Failures, Part I
Logistic Regression for Modeling Bank Failures, Part II
Logistic Regression for Modeling Bank Failures, Part III
Supervised Learning: Conclusion
Module 4 Assessment
Module 4 Project
2 - Fundamentals of Machine Learning in Finance
Неделя 1: Fundamentals of Supervised Learning in Finance
- Describe linear regression and classification models and methods of their evaluation
- Use Support Vector Machines algorithm for regression
- Use SVM to model credit spreads for illiquid names
- Train a CART Classification tree on banking data using scikit-learn
- Use Decision Trees and Random Forests to perform binary classification of bank defaults in the weekly assignment
- Compare Maching Learning to Linear Regression
- Describe the main methods and techniques of Machine Learning in Finance
Machine Learning in Finance: Review
What is Machine Learning in Finance?
Introduction to Fundamentals of Machine Learning in Finance
Lecture 1. Support Vector Machines
Support Vector Machines, Part 1
Support Vector Machines, Part 2
SVM. The Kernel Trick
Example: SVM for Prediction of Credit Spreads
Lecture 2. Supervised Learning. Tree methods
Tree Methods. CART Trees
Tree Methods: Random Forests
Tree Methods: Boosting
Module 1 Assessment
Неделя 2: Core Concepts of Unsupervised Learning, PCA and Dimensionality Reduction
- Conduct PCA and apply it to analysis of stocks
- Conduct t-Distributed Stochastic Neighbor Embedding (tSNE) method and apply it to data visualization
- Construct Eigen Portfolio via PCA in the weekly assignment
- Relate to new Python tools that can be used for a network analysis and equity correlation analysis
- Distinguish between four different ways to build representation of data in Unsupervised Learning
Core Concepts of Unsupervised Learning
Core Concepts of UL
Principal Component Analysis for Stock Returns
PCA for Stock Returns, Part 1
PCA for Stock Returns, Part 2
Dimension Reduction
Dimension Reduction with PCA
Dimension Reduction with tSNE
Module 2 Assessment
Неделя 3: Data Visualization and Clustering
- Demonstrate how clustering methods are used to DE-NOISE equity correlation matrices in Asset Management.
- Discover how to implement the online K-means with Neural Networks.
- Apply clustering techniques to the problem of estimating equity correlations from data in quantitative trading, asset management, and systematic risk monitoring.
- Apply Unsupervised Learning to modeling stock prices and stock trading.
- Compute unexpected log returns and study how individual stock returns relate to broad market index returns.
Unsupervised Learning
UL. Clustering Algorithms
UL. K-clustering
UL. K-means Neural Algorithm
UL. Hierarchical Clustering Algorithms
UL. Clustering and Estimation of Equity Correlation Matrix
UL. Minimum Spanning Trees, Kruskal Algorithm
UL. Probabilistic Clustering
G. Bonanno et. al. “Networks of equities in financial markets”, The European Physical Journal B, vol. 38, issue 2, pp. 363-371 (2004)
Module 3 Assessment
Неделя 4: Sequence Modeling and Reinforcement Learning
- Describe sequence learning with Markov models and neural networks
- Describe the concepts and algorithms of HMMs, and present financial use cases.
- Explain how RL provides more practical ways to optimal control problems in finance.
- Solve Absorption Ratio problem via PCA in the course project.
- Discover principles of dynamic programming and the Bellman equation.
Sequence Modeling
SM. Latent Variables
Sequence Modeling
SM. Latent Variables for Sequences
SM. State-Space Models
SM. Hidden Markov Models
Neural Architecture for Sequential Data
Reinforcement Learning
RL. Introduction
RL. Core Ideas
Markov Decision Process and RL
RL. Bellman Equation
RL and Inverse Reinforcement Learning
S. Marsland, “Machine Learning: an Algorithmic Perspective” (Chapman & Hall 2009), Chapter 13
C. Bishop, “Pattern Recognition and Machine Learning”, Chapter 13
Course Project
3 - Reinforcement Learning in Finance
Неделя 1: MDP and Reinforcement Learning
- Describe how Markov Decision Processes (MDP) can be used for pricing and risk management in Finance.
- Refresh and extend what we learned about MDPs and RL in the previous course
- Show how the option pricing and risk management can be formulated as Reinforcement Learning tasks
- Compare the discrete-time option pricing model with the Black-Scholes model
Markov Decision Processes (MDP)
Introduction to Markov Decision Processes and Reinforcement Learning in Finance
MDP and RL: Decision Policies
MDP & RL: Value Function and Bellman Equation
MDP & RL: Value Iteration and Policy Iteration
MDP & RL: Action Value Function
Discrete-Time Black-Scholes Model
Options and Option pricing
Black-Scholes-Merton (BSM) Model
BSM Model and Risk
Discrete Time BSM Model
Discrete Time BSM Hedging and Pricing
Discrete Time BSM BS Limit
Hedged Monte Carlo: low variance derivative pricing with objective probabilities
M. Potters, J.P. Bouchaud, and D. Sestovic, “Hedged Monte Carlo: low variance derivative pricing with objective probabilities”, https://arxiv.org/abs/cond-mat/0008147.
Неделя 2: MDP model for option pricing: Dynamic Programming Approach
- Introduce the action-value function for a discrete-time Black-Scholes model
- Explain the Bellman equation in the context of a discrete-time Black-Scholes model
- Discover how methods used on Monte Carlo and Dynamic Programming can be used to price and risk manage financial options
- Apply this knowledge, and produce a code for a Dynamic Programming solution for option pricing
MDP for Discrete-Time BS Model
MDP Formulation
Action-Value Function
Optimal Action From Q Function
Backward Recursion for Q Star
Monte-Carlo Slution
Basis Functions
Optimal Hedge With Monte-Carlo
Optimal Q Function With Monte-Carlo
Module 2 Assessment
23 - Apply Bellman principal of optimality to MDP without knowing model dynamics. By relying only on data you will solve Bellman equation by Q-learning method.
24 - Implement QLBS model which calculates optimal Q-function which gives both the optimal price and optimal hedge in time-discretized version of the Black-Scholes(-Merton) model.
Assignment files can be found in this week's Notebook. Use Jupyter Notebook Q&A for your reference.
QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds
I. Halperin, “QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds”, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3087076
I. Halperin, “The QLBS Learner Goes NuQLear: Fitted Q Iteration, Inverse RL, and Option Portfolios”,
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3102707
Course Project Reading: Global Portfolio Optimization
F. Black and R. Litterman, "Global Portfolio Optimization", Financial Analyst Journal, Sept-Oct. 1992, 28-43
Неделя 3: MDP model for option pricing - Reinforcement Learning approach
- Explain the Reinforcement Learning approach to the MDP model for option pricing
- Explain batch-mode Reinforcement Learning
- Discuss stochastic approximations
- Discover Q-learning in Finance (without Tic Tack Toe or robotics examples)
- Develop Fitted Q-iteration RL approach to option pricing in Finance
- Use Q-learning and Fitted Q Iteration for option pricing
MDP: RL Approach
Week Introduction
Batch Reinforcement Learning
Stochastic Approximations
Q-Learning
Fitted Q-Iteration
Fitted Q-Iteration: the Ψ-basis
Fitted Q-Iteration at Work
RL Solution: Discussion and Examples
Module 3 Assessment
Неделя 4: RL and INVERSE RL for Portfolio Stock Trading
- Explain how Reinforcement Learning is used for stock trading
- Explain stochastic policies
- Discover G-learning for Reinforcement Learning of optimal trading
- Discuss applications of Inverse Reinforcement Learning for finding optimal policies and rewards
- Analyze a market model derived from Inverse Reinforcement Learning
RL for Stock Trading
Week Welcome Video
Introduction to RL for Trading
Portfolio Model
One Period Rewards
Forward and Inverse Optimisation
Reinforcement Learning for Portfolios
Entropy Regularized RL
RL Equations
RL and Inverse Reinforcement Learning Solutions
Course Summary
Module 4 Assessment
Multi-period trading via Convex Optimization
S. Boyd, E. Busseti, S. Diamond, R.N. Kahn, J. Koh, P. Nystrup, and J. Speth, “Multi-period trading via Convex Optimization”, https://arxiv.org/abs/1705.00109
4 - Overview of Advanced Methods of Reinforcement Learning in Finance
Неделя 1: Black-Scholes-Merton model, Physics and Reinforcement Learning
- Understand the origins of competitive market equilibrium models in finance.
- Recognize the links between concepts in financial modeling and concepts developed in physics.
- Get familiar with popular approaches to modeling market frictions and feedback effects for option trading.
- Review (or quickly) learn Ito’s calculus, and practice in solving simple problems around option pricing and optimal control.
Lesson 1
Reinforcement Learning and Ptolemy's Epicycles
PDEs in Physics and Finance
Competitive Market Equilibrium Models in Finance
I Certainly Hope You Are Wrong, Herr Professor!
Risk as a Science of Fluctuation
Markets and the Heat Death of the Universe
Lesson 2
Option Trading and RL
Liquidity
Modeling Market Frictions
Modeling Feedback Frictions
Assignment 1
Неделя 2: Reinforcement Learning for Optimal Trading and Market Modeling
- Explore applications of RL and IRL based approaches to modeling market dynamics
- Understand the main deficiencies of the Geometric Brownian Motion (GBM) and related models as models of market dynamics.
- Introduce the Verhulst model as a model of dynamics with saturation.
- Introduce a RL-inspired non-linear model of market dynamics with frictions.
- Practice in solving the Verhulst model, and learn about it's relation with models of chaos.
Lesson 1
From Portfolio Optimization to Market Model
Invisible Hand
The GBM Model: An Unbounded Growth Without Defaults
Dynamics with Saturation: The Verhulst Model
The Singularity is Near
What are Defaults?
Quantum Equilibrium-Disequilibrium
Assignment 2
Неделя 3: Perception - Beyond Reinforcement Learning
- Introduce the ‘Quantum Equilibrium-Disequilibrium’ (QED) model of market dynamics with frictions.
- Get familiar with the Langevin equation.
- Learn basic concepts of classical mechanics.
- Introduce the Fokker-Planck equation (FPE).
- Explore methods of solving the FPE for the QED model to describe dynamics with crashes and defaults.
Lesson 1
Welcome!!
Market Dynamics and IRL
Diffusion in a Potential: The Langevin Equation
Classical Dynamics
Potential Minima and Newton's Law
Classical Dynamics: the Lagrangian and the Hamiltonian
Langevin Equation and Fokker-Planck Equations
The Fokker-Planck Equation and Quantum Mechanics
Assignment 3
Неделя 4: Other Applications of Reinforcement Learning: P-2-P Lending, Cryptocurrency, etc.
- Conceptualize the mail differences between the order-driven and quote-driven markets.
- Get familiar with the functioning of a limit order book (LOB).
- Overview statistical and ML and approaches to modeling of LOB.
- Explore the appropriateness of RL for modeling the LOB.
- Explore other potential areas for RL: P2P lending, cryptocurrency trading, etc.
Lesson 1
Welcome!!
Electronic Markets and LOB
Trades, Quotes and Order Flow
Limit Order Book
LOB Modeling
LOB Statistical Modeling
LOB Modeling with ML and RL
Other Applications of RL
The Value of Universatility
Week 4 Assessment
Final Project: Exploration of non-linear market model dynamics