Fin

https://www.coursera.org/specializations/machine-learning-reinforcement-finance

Machine Learning and Reinforcement Learning in Finance Специализация

Содержание

1 - Guided Tour of Machine Learning in Finance

• Неделя 1: Artificial Intelligence & Machine Learning
• Неделя 2: Mathematical Foundations of Machine Learning
• Неделя 3: Introduction to Supervised Learning
• Неделя 4: Supervised Learning in Finance

2 - Fundamentals of Machine Learning in Finance

• Неделя 1: Fundamentals of Supervised Learning in Finance
• Неделя 2: Core Concepts of Unsupervised Learning, PCA and Dimensionality Reduction
• Неделя 3: Data Visualization and Clustering
• Неделя 4: Sequence Modeling and Reinforcement Learning

3 - Reinforcement Learning in Finance

• Неделя 1: MDP and Reinforcement Learning
• Неделя 2: MDP model for option pricing: Dynamic Programming Approach
• Неделя 3: MDP model for option pricing - Reinforcement Learning approach
• Неделя 4: RL and INVERSE RL for Portfolio Stock Trading

4 - Overview of Advanced Methods of Reinforcement Learning in Finance

• Неделя 1: Black-Scholes-Merton model, Physics and Reinforcement Learning
• Неделя 2: Reinforcement Learning for Optimal Trading and Market Modeling
• Неделя 3: Perception - Beyond Reinforcement Learning
• Неделя 4: Other Applications of Reinforcement Learning: P-2-P Lending, Cryptocurrency, etc.

1 - Guided Tour of Machine Learning in Finance

Неделя 1: Artificial Intelligence & Machine Learning

• Get a sense of what Machine Learning is and how it relates to Artificial Intelligence.
• Recognize ML in everyday life: algorithms around us.
• Illustrate the need in ML applications.
• Explain core paradigms of ML: Supervised, Unsupervised and Reinforcement Learning.
• Compare ML for Finance with ML in Technology (image and speech recognition, robotics, etc.)

Introduction to the Specialization "Machine Learning and Reinforcement Learning in Finance"

Artificial Intelligence and Machine Learning, Part I

Artificial Intelligence and Machine Learning, Part

Artificial Intelligence and Machine Learning

Machine Learning as a Foundation of Artificial Intelligence, Part I

Machine Learning as a Foundation of Artificial Intelligence, Part II

Machine Learning as a Foundation of Artificial Intelligence, Part III

Machine Learning as a Foundation of Artificial Intelligence

Machine Learning as a Foundation of Artificial Intelligence, Part I

Machine Learning as a Foundation of Artificial Intelligence, Part II

Machine Learning as a Foundation of Artificial Intelligence, Part III

Machine Learning in Finance vs Machine Learning in Tech

Machine Learning in Finance vs Machine Learning in Tech, Part I

Machine Learning in Finance vs Machine Learning in Tech, Part II

Machine Learning in Finance vs Machine Learning in Tech, Part III

Module 1 Assessment

How AI And Automation Will Shape Finance In The Future
https://www.forbes.com/sites/workday/2017/11/03/how-ai-and-automation-will-shape-finance-in-the-future/?sh=325b7bfc481b

Неделя 2: Mathematical Foundations of Machine Learning

• Explain generalization as a goal of machine learning.
• Interpret bias-variance tradeoff in complex and simple models.
• Illustrate on example the No Free Lunch Theorem.
• Interpret a statistical method as the first ML algorithm.
• Describe how different is ML from statistical modeling.
• Choose hyperparameters using cross-validation.
• Apply methods of linear and logistic regressions learned in the course to FDIC banking call report data in the course project.

Generalization and a Bias-Variance Tradeoff

Generalization and a Bias-Variance Tradeoff

The No Free Lunch Theorem

Overfitting and Model Capacity

Linear Regression

Regularization, Validation Set, and Hyper-parameters

Overview of the Supervised Machine Learning in Finance

Module 2 Assessment

Leo Breiman, “Statistical Modeling: The Two Cultures”

Неделя 3: Introduction to Supervised Learning

• Describe sequence learning with Markov models and neural networks
• Describe the concepts and algorithms of HMMs, and present financial use cases.
• Explain how RL provides more practical ways to optimal control problems in finance.
• Solve Absorption Ratio problem via PCA in the course project.
• Discover principles of dynamic programming and the Bellman equation.

Introduction to Neural Networks and Tensor Flow

DataFlow and TensorFlow

A First Demo of TensorFlow

Linear Regression in TensorFlow

Neural Networks

Gradient Descent Optimization

Gradient Descent for Neural Networks

Stochastic Gradient Descent

E. Fama and K. French, “Size and Book-to-Market Factors in Earnings and Returns”, Journal of Finance, vol. 50, no. 1 (1995), pp. 131-155.

J. Piotroski, “Value Investing: The Use of Historical Financial Statement Information to Separate Winners from Losers”, Journal of Accounting Research, Vol. 38, Supplement: Studies on Accounting Information and the Economics of the Firm (2000), pp. 1-41

Module 3 Assessment

Неделя 4: Supervised Learning in Finance

• Develop a user case of application of Supervised Learning algorithms to finance.
• Discuss usages of ML for trading, asset management, banking risk management.
• Apply both Linear and Non-Linear, Neural Network Regression to solve the problem of prediction of Earnings Per Share.
• Use fluently TensorFlow and other packages such as Scikit-Learn and Statsmodels.

Prediction of Earning per Share (EPS) with Scikit-learn and TensorFlow

Regression and Equity Analysis

Fundamental Analysis

Machine Learning with Probabilistic Models (Classification Tasks)

Machine Learning as Model Estimation

Maximum Likelihood Estimation

Probabilistic Classification Models

Logistic Regression for Modeling Bank Failures, Part I

Logistic Regression for Modeling Bank Failures, Part II

Logistic Regression for Modeling Bank Failures, Part III

Supervised Learning: Conclusion

Module 4 Assessment

Module 4 Project

2 - Fundamentals of Machine Learning in Finance

Неделя 1: Fundamentals of Supervised Learning in Finance

• Describe linear regression and classification models and methods of their evaluation
• Use Support Vector Machines algorithm for regression
• Use SVM to model credit spreads for illiquid names
• Train a CART Classification tree on banking data using scikit-learn
• Use Decision Trees and Random Forests to perform binary classification of bank defaults in the weekly assignment
• Compare Maching Learning to Linear Regression
• Describe the main methods and techniques of Machine Learning in Finance

Machine Learning in Finance: Review

What is Machine Learning in Finance?

Introduction to Fundamentals of Machine Learning in Finance

Lecture 1. Support Vector Machines

Support Vector Machines, Part 1

Support Vector Machines, Part 2

SVM. The Kernel Trick

Example: SVM for Prediction of Credit Spreads

Lecture 2. Supervised Learning. Tree methods

Tree Methods. CART Trees

Tree Methods: Random Forests

Tree Methods: Boosting

Module 1 Assessment

Неделя 2: Core Concepts of Unsupervised Learning, PCA and Dimensionality Reduction

• Conduct PCA and apply it to analysis of stocks
• Conduct t-Distributed Stochastic Neighbor Embedding (tSNE) method and apply it to data visualization
• Construct Eigen Portfolio via PCA in the weekly assignment
• Relate to new Python tools that can be used for a network analysis and equity correlation analysis
• Distinguish between four different ways to build representation of data in Unsupervised Learning

Core Concepts of Unsupervised Learning

Core Concepts of UL

Principal Component Analysis for Stock Returns

PCA for Stock Returns, Part 1

PCA for Stock Returns, Part 2

Dimension Reduction

Dimension Reduction with PCA

Dimension Reduction with tSNE

Module 2 Assessment

Неделя 3: Data Visualization and Clustering

• Demonstrate how clustering methods are used to DE-NOISE equity correlation matrices in Asset Management.
• Discover how to implement the online K-means with Neural Networks.
• Apply clustering techniques to the problem of estimating equity correlations from data in quantitative trading, asset management, and systematic risk monitoring.
• Apply Unsupervised Learning to modeling stock prices and stock trading.
• Compute unexpected log returns and study how individual stock returns relate to broad market index returns.

Unsupervised Learning

UL. Clustering Algorithms

UL. K-clustering

UL. K-means Neural Algorithm

UL. Hierarchical Clustering Algorithms

UL. Clustering and Estimation of Equity Correlation Matrix

UL. Minimum Spanning Trees, Kruskal Algorithm

UL. Probabilistic Clustering

G. Bonanno et. al. “Networks of equities in financial markets”, The European Physical Journal B, vol. 38, issue 2, pp. 363-371 (2004)

Module 3 Assessment

Неделя 4: Sequence Modeling and Reinforcement Learning

• Describe sequence learning with Markov models and neural networks
• Describe the concepts and algorithms of HMMs, and present financial use cases.
• Explain how RL provides more practical ways to optimal control problems in finance.
• Solve Absorption Ratio problem via PCA in the course project.
• Discover principles of dynamic programming and the Bellman equation.

Sequence Modeling

SM. Latent Variables

Sequence Modeling

SM. Latent Variables for Sequences

SM. State-Space Models

SM. Hidden Markov Models

Neural Architecture for Sequential Data

Reinforcement Learning

RL. Introduction

RL. Core Ideas

Markov Decision Process and RL

RL. Bellman Equation

RL and Inverse Reinforcement Learning

S. Marsland, “Machine Learning: an Algorithmic Perspective” (Chapman & Hall 2009), Chapter 13

C. Bishop, “Pattern Recognition and Machine Learning”, Chapter 13

Course Project

3 - Reinforcement Learning in Finance

Неделя 1: MDP and Reinforcement Learning

• Describe how Markov Decision Processes (MDP) can be used for pricing and risk management in Finance.
• Refresh and extend what we learned about MDPs and RL in the previous course
• Show how the option pricing and risk management can be formulated as Reinforcement Learning tasks
• Compare the discrete-time option pricing model with the Black-Scholes model

Markov Decision Processes (MDP)

Introduction to Markov Decision Processes and Reinforcement Learning in Finance

MDP and RL: Decision Policies

MDP & RL: Value Function and Bellman Equation

MDP & RL: Value Iteration and Policy Iteration

MDP & RL: Action Value Function

Discrete-Time Black-Scholes Model

Options and Option pricing

Black-Scholes-Merton (BSM) Model

BSM Model and Risk

Discrete Time BSM Model

Discrete Time BSM Hedging and Pricing

Discrete Time BSM BS Limit

Hedged Monte Carlo: low variance derivative pricing with objective probabilities

M. Potters, J.P. Bouchaud, and D. Sestovic, “Hedged Monte Carlo: low variance derivative pricing with objective probabilities”, https://arxiv.org/abs/cond-mat/0008147.

Неделя 2: MDP model for option pricing: Dynamic Programming Approach

• Introduce the action-value function for a discrete-time Black-Scholes model
• Explain the Bellman equation in the context of a discrete-time Black-Scholes model
• Discover how methods used on Monte Carlo and Dynamic Programming can be used to price and risk manage financial options
• Apply this knowledge, and produce a code for a Dynamic Programming solution for option pricing

MDP for Discrete-Time BS Model

MDP Formulation

Action-Value Function

Optimal Action From Q Function

Backward Recursion for Q Star

Monte-Carlo Slution

Basis Functions

Optimal Hedge With Monte-Carlo

Optimal Q Function With Monte-Carlo

Module 2 Assessment

23 - Apply Bellman principal of optimality to MDP without knowing model dynamics. By relying only on data you will solve Bellman equation by Q-learning method.

24 - Implement QLBS model which calculates optimal Q-function which gives both the optimal price and optimal hedge in time-discretized version of the Black-Scholes(-Merton) model.
Assignment files can be found in this week's Notebook. Use Jupyter Notebook Q&A for your reference.

QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds
I. Halperin, “QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds”, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3087076

I. Halperin, “The QLBS Learner Goes NuQLear: Fitted Q Iteration, Inverse RL, and Option Portfolios”,
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3102707

Course Project Reading: Global Portfolio Optimization
F. Black and R. Litterman, "Global Portfolio Optimization", Financial Analyst Journal, Sept-Oct. 1992, 28-43

Неделя 3: MDP model for option pricing - Reinforcement Learning approach

• Explain the Reinforcement Learning approach to the MDP model for option pricing
• Explain batch-mode Reinforcement Learning
• Discuss stochastic approximations
• Discover Q-learning in Finance (without Tic Tack Toe or robotics examples)
• Develop Fitted Q-iteration RL approach to option pricing in Finance
• Use Q-learning and Fitted Q Iteration for option pricing

MDP: RL Approach

Week Introduction

Batch Reinforcement Learning

Stochastic Approximations

Q-Learning

Fitted Q-Iteration

Fitted Q-Iteration: the Ψ-basis

Fitted Q-Iteration at Work

RL Solution: Discussion and Examples

Module 3 Assessment

Неделя 4: RL and INVERSE RL for Portfolio Stock Trading

• Explain how Reinforcement Learning is used for stock trading
• Explain stochastic policies
• Discover G-learning for Reinforcement Learning of optimal trading
• Discuss applications of Inverse Reinforcement Learning for finding optimal policies and rewards
• Analyze a market model derived from Inverse Reinforcement Learning

RL for Stock Trading

Week Welcome Video

Introduction to RL for Trading

Portfolio Model

One Period Rewards

Forward and Inverse Optimisation

Reinforcement Learning for Portfolios

Entropy Regularized RL

RL Equations

RL and Inverse Reinforcement Learning Solutions

Course Summary

Module 4 Assessment

Multi-period trading via Convex Optimization

S. Boyd, E. Busseti, S. Diamond, R.N. Kahn, J. Koh, P. Nystrup, and J. Speth, “Multi-period trading via Convex Optimization”, https://arxiv.org/abs/1705.00109

4 - Overview of Advanced Methods of Reinforcement Learning in Finance

Неделя 1: Black-Scholes-Merton model, Physics and Reinforcement Learning

• Understand the origins of competitive market equilibrium models in finance.
• Recognize the links between concepts in financial modeling and concepts developed in physics.
• Get familiar with popular approaches to modeling market frictions and feedback effects for option trading.
• Review (or quickly) learn Ito’s calculus, and practice in solving simple problems around option pricing and optimal control.

Lesson 1

Reinforcement Learning and Ptolemy's Epicycles

PDEs in Physics and Finance

Competitive Market Equilibrium Models in Finance

I Certainly Hope You Are Wrong, Herr Professor!

Risk as a Science of Fluctuation

Markets and the Heat Death of the Universe

Lesson 2

Option Trading and RL

Liquidity

Modeling Market Frictions

Modeling Feedback Frictions

Assignment 1

Неделя 2: Reinforcement Learning for Optimal Trading and Market Modeling

• Explore applications of RL and IRL based approaches to modeling market dynamics
• Understand the main deficiencies of the Geometric Brownian Motion (GBM) and related models as models of market dynamics.
• Introduce the Verhulst model as a model of dynamics with saturation.
• Introduce a RL-inspired non-linear model of market dynamics with frictions.
• Practice in solving the Verhulst model, and learn about it's relation with models of chaos.

Lesson 1

From Portfolio Optimization to Market Model

Invisible Hand

The GBM Model: An Unbounded Growth Without Defaults

Dynamics with Saturation: The Verhulst Model

The Singularity is Near

What are Defaults?

Quantum Equilibrium-Disequilibrium

Assignment 2

Неделя 3: Perception - Beyond Reinforcement Learning

• Introduce the ‘Quantum Equilibrium-Disequilibrium’ (QED) model of market dynamics with frictions.
• Get familiar with the Langevin equation.
• Learn basic concepts of classical mechanics.
• Introduce the Fokker-Planck equation (FPE).
• Explore methods of solving the FPE for the QED model to describe dynamics with crashes and defaults.

Lesson 1

Welcome!!

Market Dynamics and IRL

Diffusion in a Potential: The Langevin Equation

Classical Dynamics

Potential Minima and Newton's Law

Classical Dynamics: the Lagrangian and the Hamiltonian

Langevin Equation and Fokker-Planck Equations

The Fokker-Planck Equation and Quantum Mechanics

Assignment 3

Неделя 4: Other Applications of Reinforcement Learning: P-2-P Lending, Cryptocurrency, etc.

• Conceptualize the mail differences between the order-driven and quote-driven markets.
• Get familiar with the functioning of a limit order book (LOB).
• Overview statistical and ML and approaches to modeling of LOB.
• Explore the appropriateness of RL for modeling the LOB.
• Explore other potential areas for RL: P2P lending, cryptocurrency trading, etc.

Lesson 1

Welcome!!

Electronic Markets and LOB

Trades, Quotes and Order Flow

Limit Order Book

LOB Modeling

LOB Statistical Modeling

LOB Modeling with ML and RL

Other Applications of RL

The Value of Universatility

Week 4 Assessment

Final Project: Exploration of non-linear market model dynamics