In this article, the first in a three part series, we will present 3 working options alphas where we use options data to predict perpetual prices. Over the three articles we will present 5 alphas which achieve a combined performance above 2 Sharpe combined and 110 bps on dollars traded (All assets have sub-5 bps trading costs) on 72h rebalance (~1.5 Sharpe at 72h rebalance, ~2 Sharpe at 1h rebalance).

In the second article, we will share 2 more alphas ranging between 1 and 2 Sharpe, and show how to combine them. Then we will analyse the signal as a fully monetizable strategy in the 3rd article. These alphas use options data to predict perpetual prices of BTC, ETH, SOL, and XRP on Binance, one of the 3 we will present today is shown below:

The alphas are fairly uncorrelated and orthogonal (not perfectly, but still very good for all being from the same dataset type) as seen in the scree plot below:

I think this series will be a real treat for readers as we present real working alpha which can actually be monetized now with no part left unexplained (other than data aggregation methods as this is very extensive, we will explain how to create the features of course though).

As with the previous article, I will not bore you with excessive writing as these are not complicated ideas and the work has already been done by myself in finding them, I will provide the feature, how to replicate it, and the performance.

Read more

In this article we will detail 2 (two!) proprietary orderbook alphas which achieve over 3 sortino when combined and over 2 Sharpe each individually. To date, there is no public literature documenting this alpha and is an entirely novel creation presented exclusively to readers of the Quant Arb Substack. We achieve the below performance as a raw signal using cross sectional z-score portfolio construction:

The Alpha

Read more

Introduction

Personally, I feel as though the whole world of options statistical arbitrage has always been fairly confusing for most people. In my view, there’s 4 main ways to approach statistical arbitrage strategies using options (that I’ve worked with at least). I’m not talking about pairs trading here for those who may be a little confused, I’m instead talking about statistical medium frequency alphas that trade options. I will give the frameworks for modelling these effects and the ways in which alpha is often discovered.

This is perhaps one of the areas most shrouded in mystery (as if doing good statistical arbitrage research wasn’t hard enough). I’ll break it down in this article the ways that I’ve always approached the problem (all of which I’ve found to be fairly successful, although some a bit more than others)

We will talk only about strategies that directly trade options, and not about strategies that use information from options markets to trade. You can do that and it works well, although for crypto options effects are a fairly weak in my opinion and I’ve always had trouble monetising the metrics I found that worked partially because you can only really trade BTC/ETH (so it’s either BTC/ETH relative value or a time series strategy, which rules out cross sectional which would be my preferred way do it if I could), but mostly because the options market in crypto is a much smaller part of the total volume compared to markets like equities where options flows are big game. Anyways, you can still use options as signals in linear stuff, but that’s a story for another time.

Read more

Introduction

Alphas make up the core of what drives a large majority of firms trading PnL. You can have complicated quoting rules, great models, amazing optimizers, advanced vol curves, or fast latency, but the gist of most trading firms edge can be compiled into some alpha representation. Even execution relies heavily on “execution alphas” in order to deliver highly effective execution costs to trading strategies. In this article, we will cover how to structure and trade alphas in the context of MFT, HFT, execution trading, arbitrage, and options (OMM + options MFT).

Out of all the ways to find an edge, one stands out the most and it is having very strong alphas signals. HFT strategies use this, MFT strategies use this, options strategies use this, execution algorithms use this - practically all advanced trading operations will have some set of proprietary alphas they use to inform their understanding of where price is going. The only exception will be arbitrage, but even that we will see becomes quite alpha like in some cases.

MFT Alphas

This is where everyone hears about alphas mostly, and when I say alphas I am specifically talking about features or “formulaic alphas”. You can generate logical alphas as well, but I find these to be very inefficient. The pipeline I will mention, is a very well known pipeline - one I have talked about in the past. You take in data, you engineer it into your features or “alphas” which is where the true profitability lives and then from here everything else serves as a performance enhancer which builds on your alphas. This is your forecasting, and then your optimizer. I like to view each block in the pipeline as a converter. We convert data into alphas (feature engineering), alphas into forecasts (forecasting / ML), forecasts into target portfolios (portfolio optimization) and target portfolios into individual trades (execution - which we will go over in the next section). The reason logical alphas are inefficient is because they bypass the portfolio optimization and forecasting stages and directly output a target portfolio in many cases. You can include them in the pipeline, but it is quite tricky to use binary outputs, and the best case is that you find a way to represent the idea as a formulaic alpha or run it separately from your formulaic pipeline (they don’t tend to like to work together and this is often what has to occur).

For MFT, it couldn’t be more clear that the alphas are the foundation. In logical form they are the whole strategy and in formulaic form they are the base of what makes all the money. If you do not have good alphas, you will find it impossible to make any money trading in any reasonably competitive market. Unless you find some horribly inefficient market (in which case finding & accessing said market is the alpha itself because such a thing is so rare!) you will rely on alphas to drive profits. Even in the case of a horribly inefficient market you will still need to have some alphas although they may be extremely basic such as quoting around Binance on a small exchange (in this case Binance’s price is the feature itself). This is a common mistake among beginner quants - they falsely believe that you can save bad features with advanced & complicated machine learning. Mentally you should view what happens after the alphas as a multiplier, ESPECIALLY machine learning as it is the most multiplier like of them all. 2 * 0 is still 0! Portfolio optimization is also important and can save you from trading yourself to death, but the boosts from here on are very incremental - and the alpha is driven by your alphas (aptly named!).

Execution

How does really high-performance execution work? Well, at some level it is very much an HFT problem, since you are trying to get limit orders filled with great markouts (hence having some HFT overlap), but at a higher timeframe it becomes an alpha problem. Say we want to do a 1h TWAP. We can choose to speed up or slow down our execution based on our view of liquidity and our view of price. Part of this is deciding the parameters of the execution algorithm itself, this is fairly simple for a market order TWAP as we can take the alpha decay of our signals we want to trade and the curve for how our execution improves as we wait longer and find the optimal intersection through interval iteration where we maximize total edge (inclusive of trading costs). From here, we can figure out the optimal number of chunks and how aggressive to be on limit orders through similar forms of analysis. This gets us to a fairly basic execution setup.

Read more

Introduction

When it comes to HFT strategies, you often reach a point where latency cannot be ignored. Simply having great alphas and modelling is not enough — you also need to be competitive on the latency front. Most of this is concentrated on the cloud network engineering front, so in today's article, we catalogue many different latency tricks.

We will dive into >10 different latency optimisations in this article to help you improve your latency setup.

Latency touches all elements of HFT strategies. For arbitrage strategies you are often competing against a couple other participants on any given market and having even a mild latency edge is often enough to thrash the competition, ensuring you aren’t left picking up their crumbs. It’s not just arbitrage strategies that need low latency, market making (which most arbitrage strategies eventually lead to since making into opportunities is the optimal approach), needs low latency in order to ensure your quotes are up to date with the latest estimates of global fair value (which a large part is just data feeds of various exchanges and getting that fast). Even statistical strategies often have latency requirements when they’re run on the HFT timescales.

This is all to say that latency is extremely important in the HFT world, and by not learning how to optimize it — you are missing out on a valuable skill which can make the difference between thrashing your competition.

Index

1. Latency Tricks:

   1. WS Rotate

   2. FIX feed

   3. Machine Gun Orders

Read more

Introduction

There is no doubt that regardless of whatever area of quant you end up in, you will end up having to do some degree of forecasting. Whether that is forecasting returns (for stat arb strategies), funding rates (for funding arb strategies), volumes (for execution strategies), or even parameters is a vol curve, it is a problem that comes up time and time again in the work of quants. Today, me and are going to be walking through our thoughts on how to do forecasting properly with some practical tips. We focus primarily on return forecasting in a statistical arbitrage manager context.

[This article is available to readers of either of our publications in it’s entirety so feel free to subscribe to either!]

Index

[Quant Arb]

1. Introduction

2. Index

3. What are we forecasting

4. Implicit Forecasts (and why they work!)

5. Models

6. Features come first

7. What doesn’t work

   1. Dropping features at the model level

   2. Dimensionality reductions

   3. Lots of bad features

8. Forecasting isn’t the best edge…

[Systematic Long Short] - On Combining Forecasts

9. The Limits Of Diversification

10. Optimal Forecast Weighting

11. When Forecast Combining Breaks Down

12. A Few Simple Heuristics…

Read more

So far we have covered how to get fair value when doing options market making, but we are yet to cover the complex parts relating to quote sizing, skewing, spreads, quoting OTC, quoting illiquids, quoting multiple exchanges, and the risks that we skew to avoid outside of the usual greeks (which are well known).

So today, in the 4th article in our options market making series, we will focus on completing the pipeline.

Read more

Introduction

Half the work with arbitrage trading is figuring out where the opportunities are. You can make your system as advanced as you want, but if you can’t find the exchanges and products where all the returns are concentrated then you won’t be very profitable.

In this article, I explain what to look for when deciding on whether to add an exchange and even when you do integrate an exchange the reasons why the initial research showing arbs could be unrealistic (wash flow, internalized flow, etc).

Then, finally — the part most of you are deeply interested in, I explicitly share where the best arbitrage opportunities currently are and can actually be captured based on my own private research.

Read more

Introduction

So you want to do market making? Perhaps you’ve seen some screenshots of those double digit Sharpe charts doing huge returns, and want to do something a bit more advanced than a pure arbitrage based approach? Well, you’re in luck. In this article, we’ll break down each component of a proper market making system and how to do it successfully.

There is obviously a large gap between doing it very well on a small exchange and doing it very well on Binance. My definition of successful will lean towards the small exchange definition since if I had alpha on Binance then I’d be making a fortune. I’ve done that in past roles (not with the same team anymore to do it again), and it’s no small feat. We had some of the best latency tech out there, plus a large team and it still took lots of work.

In today’s article, we dig into how to do market making effectively and continue some of the points from the market making for dummies article with more advanced tricks as well as recaps of simpler parts (but with expanded information on how to do them properly that didn’t cross my mind to include when writing the past article).

Read more

Introduction

Read more

Introduction

This article is a mix of various thoughts I have about how to work well with HFT data and perform high quality quantitative research. HFT data is interesting to deal with because it’s cumbersome, slow, and often messy. You want to find out what happened but you need to dig through 20 different data types from your internal logs, and on top of that often the dataset is big enough to keep your server forever if you try to process any attempt at a multi-year backtest.

Well, that’s roughly what I aim to talk about in this article. I can’t promise this will be a detailed tutorial on how to be an HFT researcher, you won’t get anywhere near that far from reading articles - in fact, you’ll need to get working with the data itself if you want to travel that far (and probably get a bit of mentorship along the way as we all tend to get), but I do think this article provides insights that can only be acquired through many years of working with the data (even if to truly become a pro you need to spend some time with the data), I would argue that a lot of this information would otherwise take ages of toiling around to figure out. Part of my professional experience has involved running research in HFT operations and as part of that I have gained insights into how to organize the research process in order for it to produce useful results. I do hope this article is useful for those in the industry who have to work with HFT data regularly. These are observations from my experience working in various HFT operations.

Read more

Introduction

Let’s start with the textbook basics - the definition. What is arbitrage afterall? Well, the dictionary will give you this definition:

the simultaneous buying and selling of securities, currency, or commodities in different markets or in derivative forms in order to take advantage of differing prices for the same asset.

I’d say this is a fairly good definition for arbitrage in the traditional sense, but in the modern sense… I’m not really sure this is true. Most advanced arbitrage trading doesn’t hedge the other side of the trade simultaneously at all. If you are in and out of the position in a couple seconds, then why would you even care about being hedged? Especially if you know that the side you are trading is the one doing all the moving.

Let’s say we have two exchanges. ShitEx and MegaEx. MegaEx trades 500 billion USD a day, ShitEx trades about 500 million USD a day. If they diverge do you really expect that MegaEx and ShitEx will both meet in the middle? Probably not… In fact, ShitEx will do almost all of the moving, which we can round to 100% of the moving when we consider that we really only care about moves that occur in excess of our cost to trade (which is very non-trivial for this kind of trading).

This is just the introduction so we’ll avoid diving into too much of the advanced weeds but you can already see from this example that the textbook definition isn’t quite what arbitrage trading looks like in practice. In today’s article, I will take a walk through all the different forms of arbitrage trading and what it realistically looks like to trade these opportunities. Not simply taking the textbook approach, but showing how real money gets made by professional arbitrageurs.

Index

1. Introduction

2. Index

3. Types of Arbitrage

4. Where can *I* find alpha?

5. Execution

6. When should you hedge?

7. When you have to predict

8. Incompletes

9. Normalization

10. Reducing trading costs

11. Trading more things

12. Leverage & Borrowing

13. Funding Arbitrage

14. Spot Arbitrage

15. Perpetuals Arbitrage

16. Triangular Arbitrage

17. Geographic Arbitrage

Read more

Introduction

First of all, this is entirely a nerd-hole, I’m not sure you should spend your time doing this as a quant. It’s a fun topic and this is a blog not a trading firm so I have the leniency to explore nerd holes but I would not spend your time on this if you want to maximize profit. If you decide that this is how you plan on making money as a fund then I am not liable for that nerd hole, but hopefully you have a fun time. That said, I find this fairly interesting and want to see where it leads and I hope the readers will also enjoy reading about the adventure. And of course, I wouldn’t be doing this if I didn’t think there is at least some edge in using AI - whether I am the guy who is going to find that edge is a different question (perhaps only Jane Street, WorldQuant, and XTX will have the time to sink into this to produce serious edge), so hey maybe we find something - but no promises.

Not just that — I also think that most people will completely mess it up and introduce some lookahead when looking at players other than the big shops - I’m talking about the academics *cough* *cough*. The issue with research that is intellectually exciting is that it is mostly popular with those who are not employed in the industry. Academics love to toy around with neural networks, complex models, and lately AI. So as a result, they lack the rigor in a lot of their (being academic - I definitely don’t get to see what the moon-shot projects are at any of the top firms sadly) research and today I hope to approach this in a much more sensible way than a lot of the papers I’ve seen.

Automated alpha discovery has been around for at least a decade, likely more. It’s a large part of WorldQuant’s investment process — amongst many other firms. Especially on the HFT timescale, the bigger firms use huge clusters to search for alphas (or just embed them inside the neural network - to be honest I don’t know the specifics, only the guys actually working there would, but we all know they like to play around with neural networks, and so far I think they’ve been fairly successful at it). Automated alpha search is something I’ve written about how to do and my views on the best way to approach the problem (I’ve had my fair share of toying around with it from the genetic approach), so feel free to check out prior articles if you want to know how it’s been done pre-AI hype (no neural networks involved!):

I can’t say I have experience working on these ML/AI teams but, I know a few people who have been a part of it and frankly things that have been done by these teams for long enough will eventually trickle down to non-AI peasants like myself who have to rely on linear regression and manual feature search to survive in this world. Outside of this though, a lot of what will inform my design of this system is really stemming from years of experience with how to make a research pipeline rigorous. Much of this article is inspired by this thread I wrote in response to a question, I will stick that in the appendix.

The Data

Everything starts with the data, and our case is no different. We will be working with data from Tardis [link] and will be making our own OHLCV bars. This will be made from the quote and trade feed. We could also give access to book data, but that will probably be quite slow to process so we will skip that (It also won’t be very useful unless we aggregate the incremental feed into at least 100 book levels. The standard 25 levels Tardis provides is not sufficient and will only be the first couple grand of liquidity for most assets).

One mistake I often see made is with how OHLC bars are created. Using trade prices for your OHLC bars creates fake mean reversion in your data since the last trade price will oscillate between the bid and the ask price. This isn’t really a lot of noise for large cap assets, but as you start going down the list of market caps (or in extremely volatile environments), then these oscillations will get very large. If you then combine this with a fixed spread cost assumption (instead of using live quote data) then you end up often finding fake mean reversion strategies. It’s one part fake mean reversion patterns from not using mid-price and another part not using the live bid/ask spread. You should make sure you fix both, but they can definitely mess up a backtest, especially on illiquid names. I talk about this a lot in my past pairs trading articles because in this case specifically you are optimizing for mean-reversion so this can easily become an issue.

Our first script is a fairly simple bit of code which takes in quote data and converts it to OHLC bars + adds data for the bid/ask price at open/close (so we can accurately simulate costs):

import pandas as pd
from tqdm import tqdm
import glob
import os

base_dir = "C:/Market_Data/Digital_Assets/Tardis_Data/Binance/Futures"

symbols = [
    'DOGEUSDT',
    'XRPUSDT',
    'LTCUSDT',
    'BNBUSDT',
    'BCHUSDT',
    'EOSUSDT',
    'ADAUSDT',
    'LINKUSDT',
    'XLMUSDT',
    'AVAXUSDT',
    'SHIBUSDT',
    'SUIUSDT',
    'DOTUSDT',
    'NEARUSDT',
    'APTUSDT',
    'UNIUSDT',
    'TONUSDT',
    'BTCUSDT',
    'ETHUSDT',
    'SOLUSDT'
]

frequencies = [
    '1min',
    '5min',
    '15min',
    '1h',
    '4h',
    '12h',
    '1d',
]

for symbol in symbols:
    data_path = os.path.join(base_dir, symbol, "quotes")

    resampled_ohlc = {freq: [] for freq in frequencies}

    file_pattern = os.path.join(data_path, f"binance-futures_quotes_*_{symbol}.csv.gz")
    files = sorted(glob.glob(file_pattern))

    for file in tqdm(files):
        df = pd.read_csv(file, compression='gzip')

        df['timestamp'] = pd.to_datetime(df['timestamp'], unit='us')
        df.set_index('timestamp', inplace=True)

        df['mid_price'] = (df['bid_price'] + df['ask_price']) / 2

        for freq in frequencies:
            resampled_mid = df['mid_price'].resample(freq, label='right', closed='right').ohlc()
            resampled_ask = df['ask_price'].resample(freq, label='right', closed='right').agg(['first', 'last'])
            resampled_bid = df['bid_price'].resample(freq, label='right', closed='right').agg(['first', 'last'])
            
            resampled = pd.concat([resampled_mid,
                                resampled_ask.rename(columns={'first': 'open_ask_price', 'last': 'close_ask_price'}),
                                resampled_bid.rename(columns={'first': 'open_bid_price', 'last': 'close_bid_price'})], axis=1)
            
            resampled_ohlc[freq].append(resampled)

    output_dir ='resampled_ohlc'
    os.makedirs(output_dir, exist_ok=True)

    for freq, df_list in resampled_ohlc.items():
        df_resampled = pd.concat(df_list)
        output_file = os.path.join(output_dir, f"{symbol}_ohlc_{freq}.parquet")
        df_resampled.to_parquet(output_file)

Then from here we add in some volume based data to increase what our AI has to work with when building its alphas:

import pandas as pd
import numpy as np
import glob
import os
from tqdm import tqdm
import gc

base_dir = "C:/Market_Data/Digital_Assets/Tardis_Data/Binance/Futures"

symbols = [
    'DOGEUSDT',
    'XRPUSDT',
    'LTCUSDT',
    'BNBUSDT',
    'BCHUSDT',
    'EOSUSDT',
    'ADAUSDT',
    'LINKUSDT',
    'XLMUSDT',
    'AVAXUSDT',
    'SHIBUSDT',
    'SUIUSDT',
    'DOTUSDT',
    'NEARUSDT',
    'APTUSDT',
    'UNIUSDT',
    'TONUSDT',
    'BTCUSDT',
    'ETHUSDT',
    'SOLUSDT'
]

frequencies = [
    '1min',
    '5min',
    '15min',
    '1h',
    '4h',
    '12h',
    '1d',
]

for symbol in symbols:
    save_dir = os.path.join('temp_trade_folders', symbol)
    os.makedirs(save_dir, exist_ok=True)
    
    trades_path = os.path.join(base_dir, symbol, 'trades')
    trade_files_pattern = os.path.join(trades_path, f"binance-futures_trades_*_{symbol}.csv.gz")
    trade_files = sorted(glob.glob(trade_files_pattern))

    existing_files = glob.glob(os.path.join(save_dir, f"*.parquet"))
    existing_dates = [pd.to_datetime(os.path.basename(file).split('_')[2]) for file in existing_files]

    for trade_file in tqdm(trade_files, desc=f"Processing trades for {symbol}"):
        trade_date = os.path.basename(trade_file).split('_')[2]
        if trade_date in existing_dates:
            continue

        df_trades = pd.read_csv(trade_file, engine='pyarrow', compression='gzip')
        df_trades['timestamp'] = pd.to_datetime(df_trades['timestamp'], unit='us')
        df_trades.set_index('timestamp', inplace=True)

        df_trades['volume'] = df_trades['price'] * df_trades['amount']
        df_trades['buy_volume'] = df_trades.apply(lambda x: x['volume'] if x['side'] == 'buy' else 0, axis=1)
        df_trades['sell_volume'] = df_trades.apply(lambda x: x['volume'] if x['side'] == 'sell' else 0, axis=1)
        df_trades['buy_trades'] = (df_trades['side'] == 'buy').astype(int)
        df_trades['sell_trades'] = (df_trades['side'] == 'sell').astype(int)

        resampled_trades = df_trades.resample('1min', label='right', closed='right').agg({
            'buy_volume': 'sum',
            'sell_volume': 'sum',
            'volume': 'sum',
            'buy_trades': 'sum',
            'sell_trades': 'sum',
            'amount': 'sum',
        })

        df_trades.to_parquet(os.path.join(save_dir, f'{symbol}_trades_{trade_date}.parquet'), engine='pyarrow')
        del df_trades, resampled_trades
        gc.collect()

for symbol in tqdm(symbols, desc="Processing symbols"):
    ohlc_path   = 'resampled_ohlc'
    ohlcv_path  = 'resampled_ohlcv'
    bars_1m_dir = os.path.join('temp_trade_folders', symbol)

    one_min_files = sorted(glob.glob(os.path.join(bars_1m_dir, '*.parquet')))

    if '1min' in frequencies:
        df_ohlc  = pd.read_parquet(f"{ohlc_path}/{symbol}_ohlc_1min.parquet")
        df_1m    = pd.concat([pd.read_parquet(f) for f in one_min_files])

        df_1m.rename(columns={'volume': 'total_volume', 'buy_trades': 'buy_trades_count', 'sell_trades': 'sell_trades_count'}, inplace=True)
        df_1m['total_trades'] = df_1m['buy_trades_count'] + df_1m['sell_trades_count']
        df_1m['vwap'] = df_1m['total_volume'] / df_1m['amount']

        df_final = df_ohlc.merge(df_1m, left_index=True, right_index=True, how='left')
        df_final.to_parquet(f"{ohlcv_path}/{symbol}_ohlcv_1min.parquet")

    slower_freqs = [f for f in frequencies if f != '1min']

    for freq in slower_freqs:
        agg = df_1m.resample(freq, label='right', closed='right').agg({
            'buy_volume'        : 'sum',
            'sell_volume'       : 'sum',
            'total_volume'      : 'sum',
            'buy_trades_count'  : 'sum',
            'sell_trades_count' : 'sum',
            'amount'            : 'sum',
        })

        # total trades and VWAP
        agg.rename(columns={'amount': 'base_volume'}, inplace=True)
        agg['total_trades'] = agg['buy_trades_count'] + agg['sell_trades_count']
        agg['vwap'] = agg['total_volume'] / agg['base_volume'].replace(0, np.nan)

        df_ohlc  = pd.read_parquet(f"{ohlc_path}/{symbol}_ohlc_{freq}.parquet")
        df_final = df_ohlc.merge(
            agg.drop(columns=['base_volume']),
            left_index=True, right_index=True, how='left'
        )

        df_final.to_parquet(f"{ohlcv_path}/{symbol}_ohlcv_{freq}.parquet")
        gc.collect()

Keep in mind that the code does hardcode in the way I have my data setup for storage with Tardis, so this may need to be changed if you like to have a different folder structure for your data. It shouldn’t be too hard — Tardis file naming / format will be the same no matter how you place it in your folders when downloading.

The Constructors

Let’s start by identifying the data types which we will feed into our system prompt:

(see appendix for text that can be copy-pasted)

Now, from here we need to come up with transforms, these are the valid operations that the system is allowed to do. We will not allow it to come up with custom ones since this could introduce lookahead bias (which it will definitely do if you let it):

(see appendix for text that can be copy-pasted)

To me these feel like a reasonable set of transforms + data types to create a wide range of strategies.

Developing The System Prompt

Now let’s try a first attempt at a prompt to generate a strategy (model=o3):

(text for prompt in appendix)

Okay, we’ve now got a first attempt at our prompt for our strategy generation agent — in all honesty, I think we will have multiple stages to this otherwise there won’t be enough diversity. I.e. we have an agent which produces a list of categories and then we loop through that to generate ideas. This should help enforce better diversity, then every so and so often maybe we let it purely go from scratch with a high temperature on for a wildcard or two. This is something we can play around with — although this is just a fun project so maybe an exercise for the reader if you really want to spend the time building intuition (which is the best way to get good at a strategy, it’s to have a gut feeling about how something will behave).

I have the above system prompt, and then after it I have the below list of frequencies, data_types, and transforms_available (which we came up with earlier in the constructors section).

Let’s try it out for a spin with our {{{length}}} set to 5, and our strategy set to the one we generated:

Nice… it looks like we’ve just built our AI hedge fund… well not really… we still need to test these alphas, record them to a database, run aggregate testing statistics of their performance, and automate all of this prompting, but hey! it’s a pretty cool step for now to know we’ve managed to generate some alphas based around an actual idea and not just automated guesswork like with genetic algorithms.

Now, digging into the alphas we can see that a lot of them try to synthetically replicate midprice by using bid_price and ask_price added together then divided by two. This is already in our dataset so it would be wise to add that in as context for the prompt.

Now, let’s try again…

When investigating the thought process this time, we can see it’s latched onto the example of using z-scores and 30-period parameter. Z-scores are great and that does give me an idea of potentially listing my “favourite transforms” or something along those lines, but for now let’s stick to an approach that enforces diversity as best possible. I also want to be doing that explicitly if I decide to, so to resolve this issue, I will add this quick line in after we give the AI the example:

***This is simply an example of how one would be formatted, you should use the input data, parameters, and transform you feel best suit the strategy.***

Running again…

Okay, looks like it is still doing the midprice thing… let’s just ban it from calculating midprice:

DO NOT USE (close_bid_price + close_ask_price) / 2 (OR THE EQUIVALENT FOR OPEN). YOU SHOULD DIRECTLY USE CLOSE OR OPEN AS THESE ARE EQUIVALENT SINCE THEY ARE THE CLOSE/OPEN MIDPRICE, NO TRADE PRICES ARE USED FOR OPEN, HIGH, LOW, OR CLOSE.

And now let’s try again:

Pretty reasonable logic, and the alphas make sense:

But, they are starting to get awfully long…

[
  {
    "frequency": "1min",
    "alpha": "scale(add(add(ts_zscore(div(sub(buy_volume,sell_volume),total_volume),20),decay_linear(delta(div(buy_trades_count,add(sell_trades_count,1)),1),5)),sign(sub(vwap,close))))"
  },
  {
    "frequency": "5min",
    "alpha": "scale(add(div(ts_sumif(sub(buy_volume,sell_volume),sub(buy_volume,sell_volume),0,10),ts_sum(abs(sub(buy_volume,sell_volume)),10)),add(delta(ts_mean(div(buy_trades_count,add(sell_trades_count,1)),5),1),ts_zscore(sub(vwap,close),10))))"
  },
  {
    "frequency": "15min",
    "alpha": "scale(add(decay_linear(div(sub(buy_volume,sell_volume),ts_mean(total_volume,10)),5),add(decay_linear(delta(div(buy_trades_count,add(sell_trades_count,1)),1),5),sign(sub(vwap,close)))))"
  },
  {
    "frequency": "1h",
    "alpha": "mul(correlation(ts_zscore(div(sub(buy_volume,sell_volume),total_volume),20),ts_zscore(sub(vwap,close),20),5),sign(delta(div(buy_trades_count,add(sell_trades_count,1)),1)))"
  },
  {
    "frequency": "4h",
    "alpha": "mul(ts_wma(div(sub(buy_volume,sell_volume),total_volume),6),mul(ts_rank(delta(div(buy_trades_count,add(sell_trades_count,1)),1),6),mul(ts_rank(inv(sub(close_ask_price,close_bid_price)),6),sign(sub(vwap,close)))))"
  }
]

I think to solve our issues relating to extra long features we will add to the prompt a ‘soft’ form of feature regularization:

Prefer simplicity wherever possible when designing your features so that we avoid overfitting.

Now the new features start to look more reasonable in their length:

[
  {
    "frequency": "1min",
    "alpha": "add(ts_zscore(div(sub(buy_volume, sell_volume), total_volume), 60), add(ts_zscore(delta(div(buy_trades_count, sell_trades_count), 1), 60), ts_zscore(sub(vwap, close), 60)))"
  },
  {
    "frequency": "5min",
    "alpha": "add(decay_linear(div(sub(buy_volume, sell_volume), total_volume), 12), add(decay_linear(delta(div(buy_trades_count, sell_trades_count), 1), 12), decay_linear(sub(vwap, close), 12)))"
  },
  {
    "frequency": "15min",
    "alpha": "add(ts_wma(div(sub(buy_volume, sell_volume), total_volume), 20), add(ts_wma(delta(div(buy_trades_count, sell_trades_count), 1), 20), ts_wma(sub(vwap, close), 20)))"
  },
  {
    "frequency": "1h",
    "alpha": "add(correlation(div(sub(buy_volume, sell_volume), total_volume), delta(div(buy_trades_count, sell_trades_count), 1), 30), ts_zscore(sub(vwap, close), 30))"
  },
  {
    "frequency": "4h",
    "alpha": "add(add(ts_rank(div(sub(buy_volume, sell_volume), total_volume), 40), ts_rank(delta(div(buy_trades_count, sell_trades_count), 1), 40)), ts_rank(sub(vwap, close), 40))"
  }
]

The reason I call this a soft form of feature regularization is we are simply suggesting in the prompt to be more mindful about the feature length without making the penalty explicit. We could use the regularization technique I discussed in my previous articles (linked at the start of this article) regarding automated alpha search, the traditional way, where we represent the feature as a tree and count the number of levels present in that tree (picture from prev. article):

We could also do something dirtier like counting the number of operators in the feature OR purely looking at the text length of the feature (maybe with the names of the input data excluded. Then if we exceed a certain threshold we would re-prompt the AI with a command to shorten the specific features that were too long, likely with a guideline of how much % shorter they needed to become. This is a bit much for now, but feel free to visit this topic yourself if you are replicating this work yourself!

Whilst doing some experimenting, I’ve realized the strategy generation prompt likes to throw in random TA indicators like ATR and RSI (in the strategy definition), but those are not in our list of transforms so I’ve now added a list of the transforms to the strategy idea generation prompt (see appendix for prompt v2).

Feature Compilation

The code which evaluates it and generates the features is in the appendix, but here is some output when I run the example function in there:

The Backtester

Now, in order to validate these alphas we also need the ability to backtest them. I’ve built a quick backtester and stuck it in the appendix.

I’ve decided to use the OHLC data because it took 14 hours for DOGE to process on volume data, and I have 19 more symbols to process and don’t plan on waiting for them to finish (and I did do some reasonable optimizations on the Pandas code). Either get a large server and multi-process or optimize it with Polars if you want it to run faster, but I think we’ll do just fine with OHLC and prove our point regardless so I’ve opted to use this data instead.

The article up until now uses the volume data though since I think better results would be found if that data is included so I will leave this open to the reader.

Wiring It All Up

The appendix has the copy-paste-able version of this, but here is the wire up:

I think this is much nicer to read compared to the Substack code insert. Here’s a quick look into the terminal output:

And of course, what do the alphas look like?

… yeah I can’t say they’re amazing, but perhaps with some better prompting, and more data than simply open, high, low, and close and 7 symbols (this is probably one of the biggest issues, but I don’t have weeks to pre-process data + infinite storage space on my laptop) then better results could be produced.

I think that’s all for today. Hope it was a fun read :)

Appendix

Feature Compiler

"""
Alpha Feature Generator

This module contains all the functions and utilities to create and evaluate
alpha features from trading data using various technical analysis operations.
"""

import numpy as np
import pandas as pd
import re

# =============================================================================
# BASIC MATHEMATICAL OPERATIONS
# =============================================================================

def add(x1: pd.DataFrame, x2: pd.DataFrame) -> pd.DataFrame:
    """Element-wise addition: x1 + x2"""
    return x1 + x2

def sub(x1: pd.DataFrame, x2: pd.DataFrame) -> pd.DataFrame:
    """Element-wise subtraction: x1 - x2"""
    return x1 - x2

def mul(x1: pd.DataFrame, x2: pd.DataFrame) -> pd.DataFrame:
    """Element-wise multiplication: x1 * x2"""
    return x1 * x2

def div(x1: pd.DataFrame, x2: pd.DataFrame) -> pd.DataFrame:
    """Protected division: x1 / x2 (handles division by zero)"""
    with np.errstate(divide='ignore', invalid='ignore'):
        result = x1 / x2
        if isinstance(result, pd.DataFrame):
            result = result.replace([np.inf, -np.inf], np.nan)
        return result

def sqrt(x1: pd.DataFrame) -> pd.DataFrame:
    """Protected square root (handles negative values)"""
    with np.errstate(invalid='ignore'):
        return np.where(x1 >= 0, np.sqrt(x1), np.sign(x1) * np.sqrt(np.abs(x1)))

def log(x1: pd.DataFrame) -> pd.DataFrame:
    """Protected logarithm (handles zero/negative values)"""
    with np.errstate(divide='ignore', invalid='ignore'):
        return np.where(x1 > 0, np.log(x1), np.sign(x1) * np.log(np.abs(x1)))

def abs(x1: pd.DataFrame) -> pd.DataFrame:
    """Absolute value"""
    return np.abs(x1)

def neg(x1: pd.DataFrame) -> pd.DataFrame:
    """Negation: -x1"""
    return -x1

def inv(x1: pd.DataFrame) -> pd.DataFrame:
    """Protected inverse: 1/x1"""
    with np.errstate(divide='ignore', invalid='ignore'):
        result = 1.0 / x1
        result = result.replace([np.inf, -np.inf], np.nan)
        return result

def max(x1: pd.DataFrame, x2: pd.DataFrame) -> pd.DataFrame:
    """Element-wise maximum"""
    return np.maximum(x1, x2)

def min(x1: pd.DataFrame, x2: pd.DataFrame) -> pd.DataFrame:
    """Element-wise minimum"""
    return np.minimum(x1, x2)

def sign(x1: pd.DataFrame) -> pd.DataFrame:
    """Sign function: returns -1, 0, or 1"""
    return np.sign(x1)

def power(x1: pd.DataFrame, x2: pd.DataFrame) -> pd.DataFrame:
    """Power function: x1^x2"""
    with np.errstate(invalid='ignore', over='ignore'):
        result = np.power(x1, x2)
        result = result.replace([np.inf, -np.inf], np.nan)
        return result

# =============================================================================
# STATISTICAL AND RANKING OPERATIONS
# =============================================================================

def rank(x1: pd.DataFrame) -> pd.DataFrame:
    """Percentile rank transformation"""
    return x1.rank(pct=True, axis=0)

def scale(x1: pd.DataFrame, a: float = 1) -> pd.DataFrame:
    """Scale by a/sum(abs(x1))"""
    a = int(a)
    sum_abs = np.nansum(np.abs(x1), axis=0)
    # Avoid division by zero
    sum_abs = np.where(sum_abs == 0, 1, sum_abs)
    return (a * x1) / sum_abs

def signedpower(x1: pd.DataFrame, a: float = 2) -> pd.DataFrame:
    """Signed power: sign(x1) * abs(x1)^a"""
    a = int(a)
    return np.sign(x1) * np.power(np.abs(x1), a)

# =============================================================================
# TIME SERIES OPERATIONS
# =============================================================================

def delay(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Lag/shift operation: x1 shifted by d periods"""
    d = max(1, int(d))
    return x1.shift(periods=d)

def delta(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Difference: x1 - x1_shifted_by_d"""
    d = max(1, int(d))
    return x1 - x1.shift(periods=d)

def ts_sum(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Rolling sum over d periods"""
    d = max(2, int(d))
    return x1.rolling(window=d, min_periods=1).sum()

def ts_mean(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Rolling mean over d periods"""
    d = max(2, int(d))
    return x1.rolling(window=d, min_periods=1).mean()

def ts_min(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Rolling minimum over d periods"""
    d = max(2, int(d))
    return x1.rolling(window=d, min_periods=1).min()

def ts_max(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Rolling maximum over d periods"""
    d = max(2, int(d))
    return x1.rolling(window=d, min_periods=1).max()

def ts_stddev(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Rolling standard deviation over d periods"""
    d = max(2, int(d))
    return x1.rolling(window=d, min_periods=2).std()

def ts_zscore(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Rolling z-score: (x1 - rolling_mean) / rolling_std over d periods"""
    d = max(2, int(d))
    rolling_mean = x1.rolling(window=d, min_periods=1).mean()
    rolling_std = x1.rolling(window=d, min_periods=2).std()
    return (x1 - rolling_mean) / rolling_std

def ts_product(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Rolling product over d periods"""
    d = max(2, int(d))
    return x1.rolling(window=d, min_periods=1).apply(lambda x: np.prod(x), raw=True)

def ts_rank(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Rank within rolling window of d periods"""
    d = max(2, int(d))
    
    def rank_last(x):
        """Rank the last value in the window"""
        if len(x) < 1:
            return np.nan
        return (x < x[-1]).sum() / len(x) + (x == x[-1]).sum() / (2 * len(x))
    
    return x1.rolling(window=d, min_periods=1).apply(rank_last, raw=True)

def ts_argmin(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Index of minimum in rolling window"""
    d = max(2, int(d))
    return x1.rolling(window=d, min_periods=1).apply(lambda x: np.argmin(x), raw=True)

def ts_argmax(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Index of maximum in rolling window"""
    d = max(2, int(d))
    return x1.rolling(window=d, min_periods=1).apply(lambda x: np.argmax(x), raw=True)

def decay_linear(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Linear weighted average over d periods"""
    d = max(2, int(d))
    weights = 2 * np.arange(1, d + 1) / (d * (d + 1))
    
    def weighted_avg(x):
        if len(x) < 1:
            return np.nan
        actual_weights = weights[-len(x):]
        actual_weights = actual_weights / actual_weights.sum()
        return np.dot(x, actual_weights)
    
    return x1.rolling(window=d, min_periods=1).apply(weighted_avg, raw=True)

def ts_wma(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Weighted moving average over d periods"""
    d = max(2, int(d))
    weights = 2 * np.arange(1, d + 1) / (d * (d + 1))
    
    def weighted_avg(x):
        if len(x) < 1:
            return np.nan
        actual_weights = weights[-len(x):]
        actual_weights = actual_weights / actual_weights.sum()
        return np.dot(x, actual_weights)
    
    return x1.rolling(window=d, min_periods=1).apply(weighted_avg, raw=True)

def ts_sma(x1: pd.DataFrame, n: int = 5, m: int = 1) -> pd.DataFrame:
    """Exponential moving average with alpha=m/n"""
    if int(n) <= 1 or int(m) <= 0:
        n, m = 5, 1
    elif m / n > 1:
        m = 1
    else:
        n, m = int(n), int(m)
    
    return x1.ewm(alpha=m/n).mean()

def ts_highday(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Days since highest value in window"""
    d = max(2, int(d))
    
    def days_since_high(x):
        if len(x) < 1:
            return np.nan
        return len(x) - 1 - np.argmax(x)
    
    return x1.rolling(window=d, min_periods=1).apply(days_since_high, raw=True)

def ts_lowday(x1: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Days since lowest value in window"""
    d = max(2, int(d))
    
    def days_since_low(x):
        if len(x) < 1:
            return np.nan
        return len(x) - 1 - np.argmin(x)
    
    return x1.rolling(window=d, min_periods=1).apply(days_since_low, raw=True)

# =============================================================================
# CORRELATION AND COVARIANCE OPERATIONS
# =============================================================================

def correlation(x1: pd.DataFrame | pd.Series, x2: pd.DataFrame | pd.Series, d: int = 5) -> pd.DataFrame | pd.Series:
    """Rolling correlation between x1 and x2"""
    d = max(2, int(d))
    
    # If both inputs are Series, return Series
    if isinstance(x1, pd.Series) and isinstance(x2, pd.Series):
        return x1.rolling(window=d, min_periods=2).corr(x2)
    
    # If either is a Series, convert to DataFrame for consistent handling
    if isinstance(x1, pd.Series):
        x1 = x1.to_frame()
    if isinstance(x2, pd.Series):
        x2 = x2.to_frame()
    
    # Ensure both dataframes have the same columns
    common_cols = x1.columns.intersection(x2.columns)
    result = pd.DataFrame(index=x1.index, columns=common_cols)
    
    for col in common_cols:
        x1_series = x1[col]
        x2_series = x2[col]
        corr = x1_series.rolling(window=d, min_periods=2).corr(x2_series)
        result[col] = corr
    
    return result

def covariance(x1: pd.DataFrame, x2: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Rolling covariance between x1 and x2"""
    d = max(2, int(d))
    
    # Ensure both dataframes have the same columns
    common_cols = x1.columns.intersection(x2.columns)
    result = pd.DataFrame(index=x1.index, columns=common_cols)
    
    for col in common_cols:
        # Calculate rolling covariance for each column
        x1_series = x1[col]
        x2_series = x2[col]
        
        # Calculate rolling covariance
        cov = x1_series.rolling(window=d, min_periods=2).cov(x2_series)
        result[col] = cov
    
    return result

# =============================================================================
# CONDITIONAL OPERATIONS
# =============================================================================

def ifcondition_g(condition_var1: pd.DataFrame, condition_var2: pd.DataFrame, 
                    x1: pd.DataFrame, x2: pd.DataFrame) -> pd.DataFrame:
    """If condition_var1 > condition_var2 then x1 else x2"""
    flag = condition_var1 > condition_var2
    result = x1.copy()
    result[~flag] = x2[~flag]
    return result

def ifcondition_ge(condition_var1: pd.DataFrame, condition_var2: pd.DataFrame,
                    x1: pd.DataFrame, x2: pd.DataFrame) -> pd.DataFrame:
    """If condition_var1 >= condition_var2 then x1 else x2"""
    flag = condition_var1 >= condition_var2
    result = x1.copy()
    result[~flag] = x2[~flag]
    return result

def ifcondition_e(condition_var1: pd.DataFrame, condition_var2: pd.DataFrame,
                    x1: pd.DataFrame, x2: pd.DataFrame) -> pd.DataFrame:
    """If condition_var1 == condition_var2 then x1 else x2"""
    flag = condition_var1 == condition_var2
    result = x1.copy()
    result[~flag] = x2[~flag]
    return result

def ts_sumif(x1: pd.DataFrame, condition_var1: pd.DataFrame, 
                condition_var2: pd.DataFrame, d: int = 5) -> pd.DataFrame:
    """Rolling sum of x1 where condition_var1 > condition_var2"""
    d = max(2, int(d))
    flag = condition_var1 > condition_var2
    masked_x1 = x1.copy()
    masked_x1[~flag] = 0
    return masked_x1.rolling(window=d, min_periods=1).sum()

def ts_count(condition_var1: pd.DataFrame, condition_var2: pd.DataFrame, 
                d: int = 5) -> pd.DataFrame:
    """Rolling count where condition_var1 > condition_var2"""
    d = max(2, int(d))
    condition = condition_var1 > condition_var2
    return condition.rolling(window=d, min_periods=1).sum()


class AlphaFeatureGenerator:
    """
    A comprehensive alpha feature generator that can evaluate complex alpha expressions
    and apply various technical analysis functions to trading data.
    """
    
    def __init__(self, alpha_expressions: list[dict[str, str]], data_types: list[str] = None):

        if data_types is None:
            data_types = [
                'open_ask_price', 'close_ask_price', 'open_bid_price', 'close_bid_price',
                'open', 'high', 'low', 'close',
                #'vwap', 'buy_volume', 'sell_volume', 'total_volume',
                #'buy_trades_count', 'sell_trades_count', 'total_trades_count',
            ]

        self.data_types = data_types
        self.alpha_expressions = alpha_expressions
    
    # =============================================================================
    # ALPHA EVALUATION ENGINE
    # =============================================================================

    def substitute_data(self, alpha: str) -> str:
        """
        Puts the data types in the alpha expression in the form of a DataFrame column.
        Uses a single regex pass to avoid conflicts.
        """
        # Sort by length (longest first) and create pattern
        sorted_data_types = sorted(self.data_types, key=len, reverse=True)
        pattern = r'\b(' + '|'.join(re.escape(dt) for dt in sorted_data_types) + r')\b'
        
        def replace_func(match):
            return f"df['{match.group(1)}']"
        
        return re.sub(pattern, replace_func, alpha)
    
    def evaluate_alpha(self, df: pd.DataFrame, alpha_expression: str) -> pd.DataFrame:
        """
        Evaluate an alpha expression on the given DataFrame.
        
        Parameters:
        -----------
        df : pd.DataFrame
            DataFrame containing the required data columns
        alpha_expression : str
            Alpha expression to evaluate
            
        Returns:
        --------
        pd.DataFrame
            Result of the alpha expression evaluation
        """
        alpha_expression = self.substitute_data(alpha_expression)

        try:
            result = eval(alpha_expression)
            
            if isinstance(result, pd.Series):
                result = result.to_frame()
            elif not isinstance(result, pd.DataFrame):
                result = pd.DataFrame(result, index=df.index)
                
            return result
            
        except Exception as e:
            print(f"Error evaluating alpha expression: {alpha_expression}")
            print(f"Error: {str(e)}")
            return pd.DataFrame()
    
    def format_output(self, alpha_result: pd.DataFrame, alpha_name: str, symbol: str) -> pd.DataFrame:
        """
        Format the alpha result into the desired output format.
        
        Parameters:
        -----------
        alpha_result : pd.DataFrame
            Raw alpha calculation result
        alpha_name : str
            Name of the alpha feature
        symbol : str
            Trading symbol/ticker
            
        Returns:
        --------
        pd.DataFrame
            Formatted output with columns: timestamp, ticker, alpha
        """
        if alpha_result.empty:
            return pd.DataFrame(columns=['timestamp', 'ticker', 'alpha'])
        
        if alpha_result.shape[1] == 1:
            alpha_values = alpha_result.iloc[:, 0]
        else:
            alpha_values = alpha_result.mean(axis=1)
        
        output = pd.DataFrame({
            'timestamp': alpha_result.index,
            'ticker': symbol,
            'alpha': alpha_values
        })
        
        output = output.dropna(subset=['alpha'])        
        output = output.reset_index(drop=True)
        
        return output
    
    def generate_feature(self, df: pd.DataFrame, feature_i: int, symbol: str) -> pd.DataFrame:
        """
        Generate alpha feature for the given DataFrame.
        
        Parameters:
        -----------
        df : pd.DataFrame
            DataFrame containing the required data columns
        feature_i : int
            Index of the feature to generate
        symbol : str
            Trading symbol/ticker
            
        Returns:
        --------
        pd.DataFrame
            Formatted alpha feature with columns: timestamp, ticker, alpha
        """
        alpha_config = self.alpha_expressions[feature_i]
        
        freq = alpha_config["frequency"]
        expression = alpha_config["alpha"]
        
        feature_name = f"alpha_{feature_i+1}_{freq}"
        print(f"Generating {feature_name}...")
        
        alpha_result = self.evaluate_alpha(df, expression)
        if not alpha_result.empty:
            formatted_output = self.format_output(alpha_result, feature_name, symbol)
            print(f"  {feature_name}: Generated {len(formatted_output)} valid data points")
            return formatted_output
        else:
            print(f"  {feature_name}: No valid data generated")
        return pd.DataFrame(columns=['timestamp', 'ticker', 'alpha'])
    
def load_data(symbol: str, freq: str):
    """
    Load data for the specified timeframe.
    """
    return pd.read_parquet(f"resampled_ohlc/{symbol}_ohlc_{freq}.parquet")    

def generate_features_for_timeframe(symbol: str, alpha_expressions: list[dict[str, str]], data_types: list[str] = None):
    """
    Generate alpha features for a specific timeframe using the load_data function.
    
    Parameters:
    -----------
    symbol : str
        Trading symbol/ticker
    alpha_expressions : list[dict[str, str]]
        List of alpha expressions to evaluate
    data_types : list[str], optional
        List of data column types to use
        
    Returns:
    --------
    dict[str, pd.DataFrame]
        Dictionary mapping alpha names to their computed values
    """
    features = {}
    generator = AlphaFeatureGenerator(alpha_expressions, data_types)
    
    for i, alpha in enumerate(alpha_expressions):
        freq = alpha["frequency"]
        df = load_data(symbol, freq)
        feature = generator.generate_feature(df, i, symbol)
        features[f"alpha_{i+1}_{freq}"] = feature
    
    return features


def example_usage():
    """Example of how to use the AlphaFeatureGenerator with load_data function"""
    
    symbol = 'BTCUSDT'
    
    # Define the alpha expressions
    alpha_expressions = [
        {
            "frequency": "5min",
            "alpha": "mul(ts_zscore(delta(close, 1), 20), ts_zscore(neg(delta(sub(close_ask_price, close_bid_price), 1)), 20))"
        },
        {
            "frequency": "5min",
            "alpha": "mul(sign(ts_zscore(delta(close, 3), 15)), sub(ts_rank(neg(delta(sub(close_ask_price, close_bid_price), 1)), 20), div(1, 2)))"
        },
        {
            "frequency": "5min",
            "alpha": "mul(ts_zscore(delta(close, 2), 30), neg(delta(div(sub(close_ask_price, close_bid_price), decay_linear(sub(close_ask_price, close_bid_price), 10)), 1)))"
        },
        {
            "frequency": "5min",
            "alpha": "correlation(ts_zscore(delta(close, 1), 20), ts_zscore(neg(delta(sub(close_ask_price, close_bid_price), 1)), 20), 10)"
        },
        {
            "frequency": "5min",
            "alpha": "mul(sign(delta(close, 2)), neg(delta(sub(close_ask_price, close_bid_price), 5)))"
        }
    ]

    
    features = generate_features_for_timeframe(symbol, alpha_expressions)
    
    return features

'''
if __name__ == "__main__":
    features = example_usage()
    
    for name, feature in features.items():
        if not feature.empty:
            print(f"\n{name}:")
            print(f"Shape: {feature.shape}")
            print(f"Columns: {feature.columns.tolist()}")
            print("Sample data:")
            print(feature.head(10))
        else:
            print(f"\n{name}: Failed to generate")
'''

Auto Hedge Fund Code

import os
import json
import uuid
from pathlib import Path
import pandas as pd
from feature_gen import generate_features
from openai import OpenAI
from backtester import backtest, load_data
import matplotlib.pyplot as plt
from matplotlib.ticker import FuncFormatter
import warnings
warnings.filterwarnings('ignore')

'''
symbols = [
    'DOGEUSDT',
    'XRPUSDT',
    'LTCUSDT',
    'BNBUSDT',
    'BCHUSDT',
    'EOSUSDT',
    'ADAUSDT',
    'LINKUSDT',
    'XLMUSDT',
    'AVAXUSDT',
    'SHIBUSDT',
    'SUIUSDT',
    'DOTUSDT',
    'NEARUSDT',
    'APTUSDT',
    'UNIUSDT',
    'TONUSDT',
    'BTCUSDT',
    'ETHUSDT',
    'SOLUSDT'
]
'''

## Using a shorter list of symbols for testing. 
symbols = [
    'ADAUSDT',
    'BTCUSDT',
    'DOGEUSDT',
    'ETHUSDT',
    'LTCUSDT',
    'SOLUSDT',
    'XRPUSDT',
]

def plot_pnl_curve(df: pd.DataFrame, feature_name: str):
    df = df.copy()
    df['timestamp'] = pd.to_datetime(df['timestamp'])
    df = df.set_index('timestamp').sort_index()

    start_val = 10_000
    equity_curve = start_val + df['cum_pnl']

    fig, ax = plt.subplots(figsize=(10, 6))
    equity_curve.plot(ax=ax)  # use default color
    ax.set_title(f"Backtest for {feature_name}", fontsize=14)
    ax.set_ylabel("Equity ($)")
    ax.set_xlabel("")  # timestamp already on x-axis

    ax.yaxis.set_major_formatter(FuncFormatter(lambda x, pos: f'${x:,.0f}'))

    for label in ax.get_xticklabels():
        label.set_rotation(45)
        label.set_horizontalalignment('right')

    ax.grid(True)
    plt.tight_layout()

    if not os.path.exists("backtest_plots"):
        os.makedirs("backtest_plots")

    file_path = f"backtest_plots/{feature_name}.png"
    fig.savefig(file_path, dpi=300)
    plt.close(fig)


def load_secrets(path: str | Path = "secrets.json") -> dict:
    """Read the secrets file and return it as a dict."""
    with Path(path).open(encoding="utf-8") as f:
        return json.load(f)
    
NO_VOLUME_DATA = True
NUM_FEATURES = 5
MODEL_CHOICE = "gpt-4o-mini"
LOOKAHEAD = 1 # How far in the future to predict.
SAMPLING_FREQ = "quarterly" # How often to sample the data for in vs out of sample testing.

secrets = load_secrets()                 
open_ai_key   = secrets.get("open_ai_key")

'''
Let's start by generating the strategy idea.
'''

idea_generation_prompt_file_name = "idea_generation_prompt_nv.txt" if NO_VOLUME_DATA else "idea_generation_prompt.txt"
idea_generation_prompt_text: str = Path(idea_generation_prompt_file_name).read_text(encoding="utf-8")

client = OpenAI(api_key=open_ai_key)    

response = client.responses.create(
    model=MODEL_CHOICE,
    input=idea_generation_prompt_text
)

strategy_idea = response.output_text

'''
Now, let's generate the alpha expressions.
'''

feature_generation_prompt_file_name = "feature_generation_prompt_nv.txt" if NO_VOLUME_DATA else "feature_generation_prompt.txt"
feature_generation_prompt_text: str = Path(feature_generation_prompt_file_name).read_text(encoding="utf-8")
feature_generation_prompt_text = feature_generation_prompt_text.replace("{{{strategy_idea}}}", strategy_idea)
feature_generation_prompt_text = feature_generation_prompt_text.replace("{{{length}}}", str(NUM_FEATURES))

client = OpenAI(api_key=open_ai_key)    

response = client.responses.create(
    model=MODEL_CHOICE,      
    input=feature_generation_prompt_text
)

alpha_expressions = response.output_text
alpha_expressions = json.loads(alpha_expressions.replace("```json", "").replace("```", ""))

'''
Now, let's generate the features in a dataframe.
'''

features = {}

for symbol in symbols:
    features[symbol] = generate_features(
        symbol=symbol,
        alpha_expressions=alpha_expressions,
    )

total_features = [pd.DataFrame() for _ in range(NUM_FEATURES)]
for i in range(NUM_FEATURES):
    for symbol in symbols:
        total_features[i] = pd.concat([total_features[i], list(features[symbol].values())[i]], axis=0)

if not os.path.exists("features"):
    os.makedirs("features")

generated_features = []
for i in range(NUM_FEATURES):
    feature_name = uuid.uuid4()
    total_features[i].to_parquet(f"features/{feature_name}.parquet", index=False)
    generated_features.append(feature_name)

'''
Time to test the features in the backtester.
'''

for alpha, feature in zip(alpha_expressions, generated_features):
    df = load_data(feature, alpha['frequency'], LOOKAHEAD, NO_VOLUME_DATA)
    pnl_curve = backtest(df, "alpha", SAMPLING_FREQ)
    plot_pnl_curve(pnl_curve, feature)

'''
From here we would probably want to generate a report with the performance statistics and the pnl curves.

I'll leave this as an exercise for the reader. If you wanted to make a full AI hedge fund you should probably stick this in a streamlit dashboard
and add a few more bells and whistles. You'll want to add a loop to do this continuously and with multiple strategies, and of course, you should
think about actually logging the strategy ideas. For now this is a proof of concept and I'll leave it as is.

'''

Backtester Code

import pandas as pd

symbols = [
    'ADAUSDT',
    'BTCUSDT',
    'DOGEUSDT',
    'ETHUSDT',
    'LTCUSDT',
    'SOLUSDT',
    'XRPUSDT',
]


def load_sim_data(freq: str = '1min', target_lookahead: int = 1, no_volume_data: bool = False) -> pd.DataFrame:
    """
    Load the simulation data for the given frequency and lookahead.
    This data will be used for calculating transaction costs and the target.
    """

    df_sim = pd.DataFrame()
    for symbol in symbols:
        df_symbol_sim = pd.read_parquet(
            f'resampled_ohlcv/{symbol}_ohlcv_{freq}.parquet' if not no_volume_data else f'resampled_ohlc/{symbol}_ohlc_{freq}.parquet',
            columns=[
                "close",
                "close_bid_price",
                "close_ask_price",
            ],
            engine='pyarrow',
        )
        df_symbol_sim.index.name = 'timestamp'
        df_symbol_sim.reset_index(inplace=True)
        df_symbol_sim['timestamp'] = pd.to_datetime(df_symbol_sim['timestamp'])
        df_symbol_sim['ticker'] = symbol
        df_symbol_sim['target'] = (
            df_symbol_sim['close']
                .pct_change(periods=target_lookahead)
                .shift(-target_lookahead)
        )
        df_symbol_sim['entry_spread_bps'] = 10_000 * ((df_symbol_sim['close_ask_price'] - df_symbol_sim['close_bid_price']) / \
                                                      ((df_symbol_sim['close_ask_price'] + df_symbol_sim['close_bid_price']) / 2)) / 2
        df_symbol_sim['exit_spread_bps'] = 10_000 * ((df_symbol_sim['close_ask_price'].shift(-target_lookahead) - df_symbol_sim['close_bid_price'].shift(-target_lookahead)) / \
                                                      ((df_symbol_sim['close_ask_price'].shift(-target_lookahead) + df_symbol_sim['close_bid_price'].shift(-target_lookahead)) / 2)) / 2

        df_symbol_sim.dropna(inplace=True)
        df_symbol_sim.drop(columns=['close'], inplace=True)
        df_sim = pd.concat([df_sim, df_symbol_sim], axis=0)
    return df_sim

def load_features(feature_name: str) -> pd.DataFrame:
    """
    Load the features for the given feature name.
    """
    df_features = pd.read_parquet(
        f'features/{feature_name}.parquet',
        engine='pyarrow',
    )
    return df_features

def merge_data(df_features: pd.DataFrame, df_sim: pd.DataFrame) -> pd.DataFrame:
    """
    Merge the features and the simulation data.
    """
    df_features = df_features.sort_values(by=['timestamp', 'ticker'])
    df_sim = df_sim.sort_values(by=['timestamp', 'ticker'])
    df_merged = pd.merge(df_features, df_sim, on=['timestamp', 'ticker'], how='left')
    df_merged.dropna(inplace=True)
    return df_merged

def load_data(
    feature_name: str,
    freq: str,
    target_lookahead: int,
    no_volume_data: bool = False,
) -> pd.DataFrame:
    """
    Load the data for the given features and target.
    """
    df_sim = load_sim_data(freq, target_lookahead, no_volume_data)
    df_features = load_features(feature_name)
    df_merged = merge_data(df_features, df_sim)
    return df_merged

def backtest(
    df: pd.DataFrame,
    feature_name: str,
    sampling_freq: str,
) -> pd.DataFrame:
    """
    Backtest the feature and the target.
    """
    
    if sampling_freq.lower() == "quarterly":
        df = df.loc[
            ((df['timestamp'].dt.quarter % 2) == (df['timestamp'].dt.year % 2))
        ]
    elif sampling_freq.lower() == "monthly":
        df = df.loc[(df['timestamp'].dt.month % 2) == (df['timestamp'].dt.year % 2)]
    else:
        raise ValueError(f"Sampling frequency {sampling_freq} not supported")
    
    df[f"position"] = 10000 * df.groupby('timestamp')[feature_name].transform(
            lambda x: (x - x.mean()) / x.sub(x.mean()).abs().sum()
    )
    df[f"pnl"] = df[f"position"] * df["target"]

    df[f"cum_pnl"] = df[f"pnl"].cumsum()

    return df

Data Types / Transforms Text

data_types = [ 
    'open', # The opening price of the period
    'high', # The highest price of the period
    'low', # The lowest price of the period
    'close', # The closing price of the period
    'vwap', # The volume-weighted average price of the period
    'buy_volume', # The sum of the buy volumes of the period
    'sell_volume', # The sum of the sell volumes of the period
    'total_volume', # The sum of the buy and sell volumes of the period
    'buy_trades_count', # The number of buy trades in the period
    'sell_trades_count', # The number of sell trades in the period
    'total_trades_count', # The number of buy and sell trades in the period
    'open_ask_price', # The opening price of the ask side of the order book (top of the book)
    'close_ask_price', # The closing price of the ask side of the order book (top of the book)
    'open_bid_price', # The opening price of the bid side of the order book (top of the book)
    'close_bid_price', # The closing price of the bid side of the order book (top of the book)
]

# Transformations to apply to the data
transforms_available = [
    'add',  # 2 inputs: (x1, x2), returns x1 + x2
    'sub',  # 2 inputs: (x1, x2), returns x1 - x2
    'mul',  # 2 inputs: (x1, x2), returns x1 * x2
    'div',  # 2 inputs: (x1, x2), returns x1 / x2 (protected division)
    'sqrt',  # 1 input: (x1), returns sqrt(x1) (protected for negative values)
    'log',  # 1 input: (x1), returns log(x1) (protected for zero/negative values)
    'abs',  # 1 input: (x1), returns absolute value of x1
    'neg',  # 1 input: (x1), returns -x1
    'inv',  # 1 input: (x1), returns 1/x1 (protected inverse)
    'max',  # 2 inputs: (x1, x2), returns element-wise maximum
    'min',  # 2 inputs: (x1, x2), returns element-wise minimum
    'rank',  # 1 input: (x1), returns percentile rank transformation
    'scale',  # 2 inputs: (x1, a), returns x1 scaled by a/sum(abs(x1))
    'signedpower',  # 2 inputs: (x1, a), returns sign(x1) * abs(x1)^a (default a=2)
    'delay',  # 2 inputs: (x1, d), returns x1 shifted by d periods
    'correlation',  # 3 inputs: (x1, x2, d), returns rolling correlation over d periods
    'covariance',  # 3 inputs: (x1, x2, d), returns rolling covariance over d periods
    'delta',  # 2 inputs: (x1, d), returns x1 - x1_shifted_by_d
    'decay_linear',  # 2 inputs: (x1, d), returns linear weighted average over d periods
    'ts_min',  # 2 inputs: (x1, d), returns rolling minimum over d periods
    'ts_max',  # 2 inputs: (x1, d), returns rolling maximum over d periods
    'ts_argmin',  # 2 inputs: (x1, d), returns index of minimum in rolling window of d periods
    'ts_argmax',  # 2 inputs: (x1, d), returns index of maximum in rolling window of d periods
    'ts_rank',  # 2 inputs: (x1, d), returns rank within rolling window of d periods
    'ts_sum',  # 2 inputs: (x1, d), returns rolling sum over d periods
    'ts_mean',  # 2 inputs: (x1, d), returns rolling mean over d periods
    'ts_product',  # 2 inputs: (x1, d), returns rolling product over d periods
    'ts_stddev',  # 2 inputs: (x1, d), returns rolling standard deviation over d periods
    'ts_zscore',  # 2 inputs: (x1, d), returns rolling z-score (x1 - rolling_mean) / rolling_std over d periods
    'ts_sma',  # 3 inputs: (x1, n, m), returns exponential moving average with alpha=m/n
    'ts_wma',  # 2 inputs: (x1, d), returns weighted moving average over d periods
    'sign',  # 1 input: (x1), returns sign of x1 (-1, 0, or 1)
    'power',  # 2 inputs: (x1, x2), returns x1^x2
    'ifcondition_g',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 > cond2 else x2
    'ifcondition_e',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 == cond2 else x2
    'ifcondition_ge',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 >= cond2 else x2
    'ts_sumif',  # 4 inputs: (x1, cond1, cond2, d), returns rolling sum of x1 where cond1 > cond2
    'ts_count',  # 3 inputs: (cond1, cond2, d), returns rolling count where cond1 > cond2
    'ts_highday',  # 2 inputs: (x1, d), returns days since highest value in window of d periods
    'ts_lowday',  # 2 inputs: (x1, d), returns days since lowest value in window of d periods
]

Prompts

This is the prompt we used to generate our strategy (v1):

Come up with a novel trading trading strategy than can be represented as a feature for an asset. Do not respond with the feature, simply the trading strategy in only 3 sentences. It should be rooted in a fundamental theory of how you view the markets and must be suitable for crypto markets. 

it must only use these input datas:

data_types = [ 
    'open', # The opening price of the period
    'high', # The highest price of the period
    'low', # The lowest price of the period
    'close', # The closing price of the period
    'vwap', # The volume-weighted average price of the period
    'buy_volume', # The sum of the buy volumes of the period
    'sell_volume', # The sum of the sell volumes of the period
    'total_volume', # The sum of the buy and sell volumes of the period
    'buy_trades_count', # The number of buy trades in the period
    'sell_trades_count', # The number of sell trades in the period
    'total_trades_count', # The number of buy and sell trades in the period
    'open_ask_price', # The opening price of the ask side of the order book (top of the book)
    'close_ask_price', # The closing price of the ask side of the order book (top of the book)
    'open_bid_price', # The opening price of the bid side of the order book (top of the book)
    'close_bid_price', # The closing price of the bid side of the order book (top of the book)
]

And (v2):

Come up with a novel trading trading strategy than can be represented as a feature for an asset. Do not respond with the feature, simply the trading strategy in only 3 sentences. It should be rooted in a fundamental theory of how you view the markets and must be suitable for crypto markets. 

it must only use these input datas:

data_types = [ 
    'open', # The opening price of the period
    'high', # The highest price of the period
    'low', # The lowest price of the period
    'close', # The closing price of the period
    'vwap', # The volume-weighted average price of the period
    'buy_volume', # The sum of the buy volumes of the period
    'sell_volume', # The sum of the sell volumes of the period
    'total_volume', # The sum of the buy and sell volumes of the period
    'buy_trades_count', # The number of buy trades in the period
    'sell_trades_count', # The number of sell trades in the period
    'total_trades_count', # The number of buy and sell trades in the period
    'open_ask_price', # The opening price of the ask side of the order book (top of the book)
    'close_ask_price', # The closing price of the ask side of the order book (top of the book)
    'open_bid_price', # The opening price of the bid side of the order book (top of the book)
    'close_bid_price', # The closing price of the bid side of the order book (top of the book)
]

and it should be able to be constructed with these transforms:

# Transformations to apply to the data
transforms_available = [
    'add',  # 2 inputs: (x1, x2), returns x1 + x2
    'sub',  # 2 inputs: (x1, x2), returns x1 - x2
    'mul',  # 2 inputs: (x1, x2), returns x1 * x2
    'div',  # 2 inputs: (x1, x2), returns x1 / x2 (protected division)
    'sqrt',  # 1 input: (x1), returns sqrt(x1) (protected for negative values)
    'log',  # 1 input: (x1), returns log(x1) (protected for zero/negative values)
    'abs',  # 1 input: (x1), returns absolute value of x1
    'neg',  # 1 input: (x1), returns -x1
    'inv',  # 1 input: (x1), returns 1/x1 (protected inverse)
    'max',  # 2 inputs: (x1, x2), returns element-wise maximum
    'min',  # 2 inputs: (x1, x2), returns element-wise minimum
    'rank',  # 1 input: (x1), returns percentile rank transformation
    'scale',  # 2 inputs: (x1, a), returns x1 scaled by a/sum(abs(x1))
    'signedpower',  # 2 inputs: (x1, a), returns sign(x1) * abs(x1)^a (default a=2)
    'delay',  # 2 inputs: (x1, d), returns x1 shifted by d periods
    'correlation',  # 3 inputs: (x1, x2, d), returns rolling correlation over d periods
    'covariance',  # 3 inputs: (x1, x2, d), returns rolling covariance over d periods
    'delta',  # 2 inputs: (x1, d), returns x1 - x1_shifted_by_d
    'decay_linear',  # 2 inputs: (x1, d), returns linear weighted average over d periods
    'ts_min',  # 2 inputs: (x1, d), returns rolling minimum over d periods
    'ts_max',  # 2 inputs: (x1, d), returns rolling maximum over d periods
    'ts_argmin',  # 2 inputs: (x1, d), returns index of minimum in rolling window of d periods
    'ts_argmax',  # 2 inputs: (x1, d), returns index of maximum in rolling window of d periods
    'ts_rank',  # 2 inputs: (x1, d), returns rank within rolling window of d periods
    'ts_sum',  # 2 inputs: (x1, d), returns rolling sum over d periods
    'ts_mean',  # 2 inputs: (x1, d), returns rolling mean over d periods
    'ts_product',  # 2 inputs: (x1, d), returns rolling product over d periods
    'ts_stddev',  # 2 inputs: (x1, d), returns rolling standard deviation over d periods
    'ts_zscore',  # 2 inputs: (x1, d), returns rolling z-score (x1 - rolling_mean) / rolling_std over d periods
    'ts_sma',  # 3 inputs: (x1, n, m), returns exponential moving average with alpha=m/n
    'ts_wma',  # 2 inputs: (x1, d), returns weighted moving average over d periods
    'sign',  # 1 input: (x1), returns sign of x1 (-1, 0, or 1)
    'power',  # 2 inputs: (x1, x2), returns x1^x2
    'ifcondition_g',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 > cond2 else x2
    'ifcondition_e',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 == cond2 else x2
    'ifcondition_ge',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 >= cond2 else x2
    'ts_sumif',  # 4 inputs: (x1, cond1, cond2, d), returns rolling sum of x1 where cond1 > cond2
    'ts_count',  # 3 inputs: (cond1, cond2, d), returns rolling count where cond1 > cond2
    'ts_highday',  # 2 inputs: (x1, d), returns days since highest value in window of d periods
    'ts_lowday',  # 2 inputs: (x1, d), returns days since lowest value in window of d periods
]

no volume version:

Come up with a novel trading trading strategy than can be represented as a feature for an asset. Do not respond with the feature, simply the trading strategy in only 3 sentences. It should be rooted in a fundamental theory of how you view the markets and must be suitable for crypto markets. 

it must only use these input datas:

data_types = [ 
    'open', # The opening price of the period
    'high', # The highest price of the period
    'low', # The lowest price of the period
    'close', # The closing price of the period
    'open_ask_price', # The opening price of the ask side of the order book (top of the book)
    'close_ask_price', # The closing price of the ask side of the order book (top of the book)
    'open_bid_price', # The opening price of the bid side of the order book (top of the book)
    'close_bid_price', # The closing price of the bid side of the order book (top of the book)
]

and it should be able to be constructed with these transforms:

# Transformations to apply to the data
transforms_available = [
    'add',  # 2 inputs: (x1, x2), returns x1 + x2
    'sub',  # 2 inputs: (x1, x2), returns x1 - x2
    'mul',  # 2 inputs: (x1, x2), returns x1 * x2
    'div',  # 2 inputs: (x1, x2), returns x1 / x2 (protected division)
    'sqrt',  # 1 input: (x1), returns sqrt(x1) (protected for negative values)
    'log',  # 1 input: (x1), returns log(x1) (protected for zero/negative values)
    'abs',  # 1 input: (x1), returns absolute value of x1
    'neg',  # 1 input: (x1), returns -x1
    'inv',  # 1 input: (x1), returns 1/x1 (protected inverse)
    'max',  # 2 inputs: (x1, x2), returns element-wise maximum
    'min',  # 2 inputs: (x1, x2), returns element-wise minimum
    'rank',  # 1 input: (x1), returns percentile rank transformation
    'scale',  # 2 inputs: (x1, a), returns x1 scaled by a/sum(abs(x1))
    'signedpower',  # 2 inputs: (x1, a), returns sign(x1) * abs(x1)^a (default a=2)
    'delay',  # 2 inputs: (x1, d), returns x1 shifted by d periods
    'correlation',  # 3 inputs: (x1, x2, d), returns rolling correlation over d periods
    'covariance',  # 3 inputs: (x1, x2, d), returns rolling covariance over d periods
    'delta',  # 2 inputs: (x1, d), returns x1 - x1_shifted_by_d
    'decay_linear',  # 2 inputs: (x1, d), returns linear weighted average over d periods
    'ts_min',  # 2 inputs: (x1, d), returns rolling minimum over d periods
    'ts_max',  # 2 inputs: (x1, d), returns rolling maximum over d periods
    'ts_argmin',  # 2 inputs: (x1, d), returns index of minimum in rolling window of d periods
    'ts_argmax',  # 2 inputs: (x1, d), returns index of maximum in rolling window of d periods
    'ts_rank',  # 2 inputs: (x1, d), returns rank within rolling window of d periods
    'ts_sum',  # 2 inputs: (x1, d), returns rolling sum over d periods
    'ts_mean',  # 2 inputs: (x1, d), returns rolling mean over d periods
    'ts_product',  # 2 inputs: (x1, d), returns rolling product over d periods
    'ts_stddev',  # 2 inputs: (x1, d), returns rolling standard deviation over d periods
    'ts_zscore',  # 2 inputs: (x1, d), returns rolling z-score (x1 - rolling_mean) / rolling_std over d periods
    'ts_sma',  # 3 inputs: (x1, n, m), returns exponential moving average with alpha=m/n
    'ts_wma',  # 2 inputs: (x1, d), returns weighted moving average over d periods
    'sign',  # 1 input: (x1), returns sign of x1 (-1, 0, or 1)
    'power',  # 2 inputs: (x1, x2), returns x1^x2
    'ifcondition_g',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 > cond2 else x2
    'ifcondition_e',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 == cond2 else x2
    'ifcondition_ge',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 >= cond2 else x2
    'ts_sumif',  # 4 inputs: (x1, cond1, cond2, d), returns rolling sum of x1 where cond1 > cond2
    'ts_count',  # 3 inputs: (cond1, cond2, d), returns rolling count where cond1 > cond2
    'ts_highday',  # 2 inputs: (x1, d), returns days since highest value in window of d periods
    'ts_lowday',  # 2 inputs: (x1, d), returns days since lowest value in window of d periods
]

This is the final prompt we ended up with for turning strategy ideas into features:

Your task is to formulate alphas, these are features created from price data. Your strategy you are building is:

{{{strategy}}}

You must select a frequency from the frequencies list, and then construct a feature which uses only the transforms and the input data types to form the feature. It must be constructed using Python format. You will return it as a json format with two fields ['frequency', 'alpha']. 'frequency' is the frequency you select from the below frequencies, and 'alpha' is the formulaic alpha. The alpha must only use input data types from the list of inputs and transforms from the transforms list otherwise it will be rejected. 

An example which is the 30 day rolling z-score of (high-low)/vwap:

{
    'frequency': '1d',
    'alpha' : 'ts_zscore(div(sub(high, low), close), 30)',
}

This is simply an example of how one would be formatted, you should use the input data, parameters, and transform you feel best suit the strategy.

You should return a list of JSONs of length {{{length}}} where each JSON is a variation of how you could formulate the strategy. This could mean simple parameter differences, or whole logic differences.

TO REPEAT IT SHOULD BE A LIST OF JSONs of LENGTH {{{length}}}. IT SHOULD BE {{{length}}} LONG. ENSURE THIS IS THE CASE

Prefer simplicity wherever possible when designing your features so that we avoid overfitting.

DO NOT USE (close_bid_price + close_ask_price) / 2 (OR THE EQUIVALENT FOR OPEN). YOU SHOULD DIRECTLY USE CLOSE OR OPEN AS THESE ARE EQUIAVELANT SINCE THEY ARE THE CLOSE/OPEN MIDPRICE, NO TRADE PRICES ARE USED FOR OPEN, HIGH, LOW, OR CLOSE.

Do not respond in a way that acknowledges the prompt. You must simply follow the prompt. Here's some examples of what NOT to do and what to do:

For an example of what NOT to do:
"Sure, here's an alpha about... :

"{
        "frequency": "5min",
        "alpha": "ts_correlation(close, high, 30)"
}"

For an example of what to do:
"{
        "frequency": "5min",
        "alpha": "ts_correlation(close, high, 30)"
}"

You also should not respond with an explanation and should only answer with the JSON. Your output will be parsed, and if you respond with anything other than a JSON it will break the parser.

# Frequencies your data will be at 
frequencies = [
    '1min',
    '5min',
    '15min',
    '1h',
    '4h',
    '12h',
    '1d',
]

# The data types available for use. All OHLC prices are formed from midprices so the close is the close midprice, as well as the same for open, high, and low.
data_types = [ 
    'open', # The opening price of the period
    'high', # The highest price of the period
    'low', # The lowest price of the period
    'close', # The closing price of the period
    'vwap', # The volume-weighted average price of the period
    'buy_volume', # The sum of the buy volumes of the period
    'sell_volume', # The sum of the sell volumes of the period
    'total_volume', # The sum of the buy and sell volumes of the period
    'buy_trades_count', # The number of buy trades in the period
    'sell_trades_count', # The number of sell trades in the period
    'total_trades_count', # The number of buy and sell trades in the period
    'open_ask_price', # The opening price of the ask side of the order book (top of the book)
    'close_ask_price', # The closing price of the ask side of the order book (top of the book)
    'open_bid_price', # The opening price of the bid side of the order book (top of the book)
    'close_bid_price', # The closing price of the bid side of the order book (top of the book)
]

# Transformations to apply to the data
transforms_available = [
    'add',  # 2 inputs: (x1, x2), returns x1 + x2
    'sub',  # 2 inputs: (x1, x2), returns x1 - x2
    'mul',  # 2 inputs: (x1, x2), returns x1 * x2
    'div',  # 2 inputs: (x1, x2), returns x1 / x2 (protected division)
    'sqrt',  # 1 input: (x1), returns sqrt(x1) (protected for negative values)
    'log',  # 1 input: (x1), returns log(x1) (protected for zero/negative values)
    'abs',  # 1 input: (x1), returns absolute value of x1
    'neg',  # 1 input: (x1), returns -x1
    'inv',  # 1 input: (x1), returns 1/x1 (protected inverse)
    'max',  # 2 inputs: (x1, x2), returns element-wise maximum
    'min',  # 2 inputs: (x1, x2), returns element-wise minimum
    'rank',  # 1 input: (x1), returns percentile rank transformation
    'scale',  # 2 inputs: (x1, a), returns x1 scaled by a/sum(abs(x1))
    'signedpower',  # 2 inputs: (x1, a), returns sign(x1) * abs(x1)^a (default a=2)
    'delay',  # 2 inputs: (x1, d), returns x1 shifted by d periods
    'correlation',  # 3 inputs: (x1, x2, d), returns rolling correlation over d periods
    'covariance',  # 3 inputs: (x1, x2, d), returns rolling covariance over d periods
    'delta',  # 2 inputs: (x1, d), returns x1 - x1_shifted_by_d
    'decay_linear',  # 2 inputs: (x1, d), returns linear weighted average over d periods
    'ts_min',  # 2 inputs: (x1, d), returns rolling minimum over d periods
    'ts_max',  # 2 inputs: (x1, d), returns rolling maximum over d periods
    'ts_argmin',  # 2 inputs: (x1, d), returns index of minimum in rolling window of d periods
    'ts_argmax',  # 2 inputs: (x1, d), returns index of maximum in rolling window of d periods
    'ts_rank',  # 2 inputs: (x1, d), returns rank within rolling window of d periods
    'ts_sum',  # 2 inputs: (x1, d), returns rolling sum over d periods
    'ts_mean',  # 2 inputs: (x1, d), returns rolling mean over d periods
    'ts_product',  # 2 inputs: (x1, d), returns rolling product over d periods
    'ts_stddev',  # 2 inputs: (x1, d), returns rolling standard deviation over d periods
    'ts_zscore',  # 2 inputs: (x1, d), returns rolling z-score (x1 - rolling_mean) / rolling_std over d periods
    'ts_sma',  # 3 inputs: (x1, n, m), returns exponential moving average with alpha=m/n
    'ts_wma',  # 2 inputs: (x1, d), returns weighted moving average over d periods
    'sign',  # 1 input: (x1), returns sign of x1 (-1, 0, or 1)
    'power',  # 2 inputs: (x1, x2), returns x1^x2
    'ifcondition_g',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 > cond2 else x2
    'ifcondition_e',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 == cond2 else x2
    'ifcondition_ge',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 >= cond2 else x2
    'ts_sumif',  # 4 inputs: (x1, cond1, cond2, d), returns rolling sum of x1 where cond1 > cond2
    'ts_count',  # 3 inputs: (cond1, cond2, d), returns rolling count where cond1 > cond2
    'ts_highday',  # 2 inputs: (x1, d), returns days since highest value in window of d periods
    'ts_lowday',  # 2 inputs: (x1, d), returns days since lowest value in window of d periods
]

no volume version:

Your task is to formulate alphas, these are features created from price data. Your strategy you are building is:

{{{strategy}}}

You must select a frequency from the frequencies list, and then construct a feature which uses only the transforms and the input data types to form the feature. It must be constructed using Python format. You will return it as a json format with two fields ['frequency', 'alpha']. 'frequency' is the frequency you select from the below frequencies, and 'alpha' is the formulaic alpha. The alpha must only use input data types from the list of inputs and transforms from the transforms list otherwise it will be rejected. 

An example which is the 30 day rolling z-score of (high-low)/close:

{
    'frequency': '1d',
    'alpha' : 'ts_zscore(div(sub(high, low), close), 30)',
}

This is simply an example of how one would be formatted, you should use the input data, parameters, and transform you feel best suit the strategy.

You should return a list of JSONs of length {{{length}}} where each JSON is a variation of how you could formulate the strategy. This could mean simple parameter differences, or whole logic differences.

TO REPEAT IT SHOULD BE A LIST OF JSONs of LENGTH {{{length}}}. IT SHOULD BE {{{length}}} LONG. ENSURE THIS IS THE CASE

Prefer simplicity wherever possible when designing your features so that we avoid overfitting.

DO NOT USE (close_bid_price + close_ask_price) / 2 (OR THE EQUIVALENT FOR OPEN). YOU SHOULD DIRECTLY USE CLOSE OR OPEN AS THESE ARE EQUIAVELANT SINCE THEY ARE THE CLOSE/OPEN MIDPRICE, NO TRADE PRICES ARE USED FOR OPEN, HIGH, LOW, OR CLOSE.

Do not respond in a way that acknowledges the prompt. You must simply follow the prompt. Here's some examples of what NOT to do and what to do:

For an example of what NOT to do:
"Sure, here's an alpha about... :

"{
        "frequency": "5min",
        "alpha": "ts_correlation(close, high, 30)"
}"

For an example of what to do:
"{
        "frequency": "5min",
        "alpha": "ts_correlation(close, high, 30)"
}"

You also should not respond with an explanation and should only answer with the JSON. Your output will be parsed, and if you respond with anything other than a JSON it will break the parser.

# Frequencies your data will be at 
frequencies = [
    '1min',
    '5min',
    '15min',
    '1h',
    '4h',
    '12h',
    '1d',
]

# The data types available for use. All OHLC prices are formed from midprices so the close is the close midprice, as well as the same for open, high, and low.
data_types = [ 
    'open', # The opening price of the period
    'high', # The highest price of the period
    'low', # The lowest price of the period
    'close', # The closing price of the period
    'open_ask_price', # The opening price of the ask side of the order book (top of the book)
    'close_ask_price', # The closing price of the ask side of the order book (top of the book)
    'open_bid_price', # The opening price of the bid side of the order book (top of the book)
    'close_bid_price', # The closing price of the bid side of the order book (top of the book)
]

# Transformations to apply to the data
transforms_available = [
    'add',  # 2 inputs: (x1, x2), returns x1 + x2
    'sub',  # 2 inputs: (x1, x2), returns x1 - x2
    'mul',  # 2 inputs: (x1, x2), returns x1 * x2
    'div',  # 2 inputs: (x1, x2), returns x1 / x2 (protected division)
    'sqrt',  # 1 input: (x1), returns sqrt(x1) (protected for negative values)
    'log',  # 1 input: (x1), returns log(x1) (protected for zero/negative values)
    'abs',  # 1 input: (x1), returns absolute value of x1
    'neg',  # 1 input: (x1), returns -x1
    'inv',  # 1 input: (x1), returns 1/x1 (protected inverse)
    'max',  # 2 inputs: (x1, x2), returns element-wise maximum
    'min',  # 2 inputs: (x1, x2), returns element-wise minimum
    'rank',  # 1 input: (x1), returns percentile rank transformation
    'scale',  # 2 inputs: (x1, a), returns x1 scaled by a/sum(abs(x1))
    'signedpower',  # 2 inputs: (x1, a), returns sign(x1) * abs(x1)^a (default a=2)
    'delay',  # 2 inputs: (x1, d), returns x1 shifted by d periods
    'correlation',  # 3 inputs: (x1, x2, d), returns rolling correlation over d periods
    'covariance',  # 3 inputs: (x1, x2, d), returns rolling covariance over d periods
    'delta',  # 2 inputs: (x1, d), returns x1 - x1_shifted_by_d
    'decay_linear',  # 2 inputs: (x1, d), returns linear weighted average over d periods
    'ts_min',  # 2 inputs: (x1, d), returns rolling minimum over d periods
    'ts_max',  # 2 inputs: (x1, d), returns rolling maximum over d periods
    'ts_argmin',  # 2 inputs: (x1, d), returns index of minimum in rolling window of d periods
    'ts_argmax',  # 2 inputs: (x1, d), returns index of maximum in rolling window of d periods
    'ts_rank',  # 2 inputs: (x1, d), returns rank within rolling window of d periods
    'ts_sum',  # 2 inputs: (x1, d), returns rolling sum over d periods
    'ts_mean',  # 2 inputs: (x1, d), returns rolling mean over d periods
    'ts_product',  # 2 inputs: (x1, d), returns rolling product over d periods
    'ts_stddev',  # 2 inputs: (x1, d), returns rolling standard deviation over d periods
    'ts_zscore',  # 2 inputs: (x1, d), returns rolling z-score (x1 - rolling_mean) / rolling_std over d periods
    'ts_sma',  # 3 inputs: (x1, n, m), returns exponential moving average with alpha=m/n
    'ts_wma',  # 2 inputs: (x1, d), returns weighted moving average over d periods
    'sign',  # 1 input: (x1), returns sign of x1 (-1, 0, or 1)
    'power',  # 2 inputs: (x1, x2), returns x1^x2
    'ifcondition_g',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 > cond2 else x2
    'ifcondition_e',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 == cond2 else x2
    'ifcondition_ge',  # 4 inputs: (cond1, cond2, x1, x2), returns x1 if cond1 >= cond2 else x2
    'ts_sumif',  # 4 inputs: (x1, cond1, cond2, d), returns rolling sum of x1 where cond1 > cond2
    'ts_count',  # 3 inputs: (cond1, cond2, d), returns rolling count where cond1 > cond2
    'ts_highday',  # 2 inputs: (x1, d), returns days since highest value in window of d periods
    'ts_lowday',  # 2 inputs: (x1, d), returns days since lowest value in window of d periods
]

Conversation

Below is some excerpts from a conversation I had on Twitter which inspired the article. As I went through my own logic of how to do it I realized this might not a total disaster of an idea and by the end of it was very tempted to have a go at it (so here we are today).

==== Quant Arb ====
Hi, 

Quick thoughts :

ChatGPT produces what I would call “intern-like” responses to a lot of quant questions.

It will cause lookahead and then get excited and think this should go into production.

It also has a very surface level understanding of things that lacks nuance.

It’s more of a “let’s put a neural network on momentum with some vague ML” than understanding characteristics where momentum works (low volatility, low volume, low chatter activity—if you have sentiment data, smooth prices).

It doesn’t want to think fundamentally about the problem and it has no concept of good research practices.

I think to avoid this you could have it plug in the alphas as a formula as well as parameters and then have the portfolio optimisation, backtest, forecasting, etc literally just be a dropdown esque parameterisation, but really you’d need to do 3 core things:

Not let it backtest anything itself (this will always be fucked)

Not let it do any ML and just create features (if you let it do ML it will be trigger happy like any intern. Let it pick from a couple options and that’s your ml— ie ridge, boosted tree, GAM). All interns go ham on ML if you let them

Not letting it do portfolio optimisation. It will 100% screw this up.

Alternatively make it use ranked long short because it’s already getting complex, and we can upgrade it later (id recommend this, leave the fancy stuff for later)

Then from there you need some metric of how many tests have been run etc because I also wouldn’t trust it not to overfit so give it some KPIs on this — QuantConnect has something like this on their platform.

And then from there you need to get it to come up with ideas about markets it can refine. It needs to have ideas about behaviour. Perhaps try to pipeline it from 1) idea 2) feature construction 3) the system takes over because we don’t trust it for anything else

That said, automated feature search is already a thing - I wrote a couple articles about it on my blog. This is nothing new and has been around like a decade so I don’t think it’s implausible that ChatGPT could do features but I wouldn’t trust it beyond that.

Lmk if you try it, keep it simple and don’t trust ChatGPT to behave itself with data practices — it will break it if you let it

==== Quant Arb ====

Alternatively you could have ChatGPT’s view on events as a feature itself and try to model that but of course part of your dataset may be in ChatGPT’s memory so be careful with that etc etc

==== Klass (Macro Arb) ====

Yeah this is an annoying obstacle for back of the envelope mid-freq backtests with an implementation like that: persistent lookahead

Will flag if anything interesting.

==== Quant Arb ==== 

You can use older GPT models. You've got at least a couple years of data here. 

Then you can compare if the performance drops off in recent years. 

There's a few papers that have done this - some good, some not so good.

==== Quant Arb ====

One of the reasons having GPT come up with alphas is nice is because there's very little chance it knows how 

zscore(fracdiff(close), 30_day) / sentiment_5d 

performed historically even if it does know how the prices performed -- even then you can compare vs data that is after train date. 

So especially if it is delta neutral and the features abstract away the price movement it should be fine

==== Quant Arb ====

As opposed to turning chatgpts views into an alpha ie

management_rating: 
is_this_place_hell_to_work_at: 
etc 
etc

as fields which chatgpt decides what it thinks the score is based on whatever your prompt is for said score

==== Quant Arb ====

Or even more plainly having it pick stocks which is most likely to get you in lookahead jail

==== Klass (Macro Arb) ====

This is really interesting actually yeah I didn't think of it like this but I can defo see how adding a layer or two of abstraction eliminates a fairly significant portion of lookahead

essentially when you ask "pick what stock you think will do well", the shortest distance to the answer will be "what stock does my memory say did really well" so 99% of the time it defaults to that

whereas when you ask "what features do you think should be a good predictor of stock returns" a much more parsimonious path to answering it is the earnest one

==== Quant Arb ====

I mean even with something like a management score if a company then went bankrupt like enron did because the management were horrid then it would give it a terrible score BUT most analysts at the time quite liked enron, their stock performed great and they looked like they made money. So I suppose you'd need to validate it against out of sample stuff, and check returns are consistent between in sample and out of sample.

But yeah pure features is pretty near impossible for it to mess up although for big name strats like momentum theres a good chance it will know the performance of that but beyond that it's probably out of it's reach

==== Quant Arb ====

Also you can't just say what features should I do because it will copy paste literature. You need some novelty in the prompt engineering.

Ie come up with a theory about participants and crowding.

Thats stage one, then stage 2 is saying how do we make this into an alpha. Give a few examples.

Any param tuning for lookback/ settings can be done automatically with an optimizer and with explicit lookahead controls, don't let it do this part

==== Quant Arb ====

If you say "give me features that make money" it'll give you the dumbest shit ever because it will regurgitate what is already well known (basic momentum, reversion, etc)

You have to make it come up with a whole idea process and then finally put that idea into feature. 

You really are generating theories about market behaviour and effects then making features. Not directly asking it for them.

Introduction

Note: The full Rust implementation of the quoter can be found in the appendix

So we’ve looked a lot at theoretical volatility curves, but this hasn’t yet been translated into something we can derive quotes from in real time. So today, we will write out a full pricing server. The server will spit out our delta neutral quotes. This is the first half of building a proper options market making system. It’s by far the easier part in terms of developer work, but does involve a lot of research and code still.

My model for developing HFT systems has always been that you have your pricing server and your trader server. Sometimes the pricing server is simply there to stream data and doesn’t do anything other than that, but often you use it to calculate fair values or in the case of taker strategies, different opportunities. Today, we will be building the first half of that - building a full options market maker is a very large lift and probably unrealistic for a single article. That said, this will still be a fairly large amount of code and a longer article as a result. Our system prices options and comes up with it’s own fair values without simply copying the market. If spot prices change, we have our own new option prices before the market updates. If a trade happens, we can price it into our curve as well. Probably nowhere near as good as other MMs, but that can be tuned in production.

We will assume that we have an empty portfolio for our quotes so there will be no skewing involved, but in a real system we would skew our quotes based on various risk factors (Greeks, notional limits, etc).

The hardest part of building an options market making system is the portfolio management, OEMS, risk management, reporting, controllers, data robustness handlers. There’s a world of difference between spitting out quotes in a system that doesn't have any sort of complex data quality checking logic + connection assurances + low latency optimizations for the data feeds, and what we will be doing. That said, the code below is by no means trivial and will involve us putting together components from all the prior articles as well as including many new additions.

Here is what some of the quotes look like when running live. You can see some of the fit parameters to the model, and then also the log-moneyness vs. the implied volatility for each of the strikes on the expiry:

If we were to plot the values it would look similar to the curve we fit in the previous article on option market making:

The only difference is that this is being done extremely fast and live. We use the same Python code under the hood as this uses SciPy’s minimize which is made of super fast FORTRAN anyways so the optimization time is still fairly acceptable. We also don’t need to fit too aggressively.

Further Work

As I mentioned earlier, this is not a full option market maker — merely the quoting component, we would need to implement execution to bring it live.

It also likely makes sense to incorporate VCR, SCR, SSR, etc into the model by using spot price updates to modify the curve in-between fits. I’ll show you how to do that live in the next article. The current implementation has the functions to do it implemented, but the spot feed has not been fit to yet.

From here, we would add on our skews our view on where the curve is going based on various fits and skews for different Greek preferences and we would have a half decent start at being an options market maker.

Now that we have this live code, I will use this as a platform in the next articles to show how this will be done in a live environment. I’ll also put down a quick dashboard so we can visualize it.

If you want to have a toy with it, the code is below.

Appendix

Here is the code for the quoter, to start here is the file directory overview:

options_pricing
├── python
│   └── wing_model.py
└── src
    ├── exchanges
    │   ├── deribit.rs
    │   └── mod.rs
    ├── volatility
    │   └── wing.rs
    ├── main.rs
    ├── model.rs
    ├── py_utils.rs
    ├── quoter.rs
    ├── ticker_universe.rs
    └── wing_model.rs
Cargo.toml
.gitignore

It should look like this when it connects to the exchanges:

And here are the fitted surfaces which happen live:

main.rs:

// Optional: Remove this, only so it doesn't crowd my terminal.
#![allow(unused_variables, unreachable_code, unused_imports, deprecated, dead_code, unused_mut)]

pub mod ticker_universe;
pub mod model;
pub mod quoter;
pub mod exchanges;

pub mod py_utils;
pub mod wing_model;
pub mod volatility {
    pub mod wing;
}

pub use exchanges::{
    ExchangeClient,
    DeribitClient,
};

use color_eyre::eyre::Error;
use model::Exchange;
use quoter::Quoter;
use ticker_universe::TickerUniverse;

#[tokio::main]
async fn main() -> Result<(), Error> {
    tracing_subscriber::fmt()
        .with_max_level(tracing::Level::DEBUG)
        .init(); 

    let client: DeribitClient = DeribitClient::connect_mainnet().await?;
    let instruments = client.get_instruments(None, "option", Some(true), None).await?; 
    let universe = TickerUniverse::new(instruments, Exchange::Deribit);
    let mut quoter = Quoter::new(Exchange::Deribit, universe, client);
    quoter.start_quoting().await.expect("Error");
    loop {};
    Ok(())
}

Read more

Read more

Often you can find an edge where others have made assumptions simply by reading the contract specification carefully. Today, we’ll walk through one example of this which causes options market makers to misprice their options. There is also the explanation that they are fully aware of this, but can’t be bothered to develop a model to fix it since this is a rather small mispricing, but it’s a clear case where options trade at a quarter to a third of their value in some cases.

This will be a short article as I’ve already written my main article for the day, and the effect doesn’t really need a deep dive into options pricing. It’s an explanation of where the mismatch is and how to correctly price it… not much more needed so it’s a short article.

How it works

With options on crypto exchanges, they all settle to a TWAP for the major exchanges. However, almost all market makers will treat them as if they expire at the last trade price so they get more and more mispriced as they head towards expiry.

A lot of market makers will factor in the accumulated prices so far since the exchange will have a feed for the expected settlement price which most MMs will use as their underlying price for the option, and this expected settlement price will factor in the TWAP value so far so this part is factored in and there is no trade here.

However, when you settle to a TWAP the volatility of the option is effectively halved. This is a rough proxy but based on my testing I found it usually ends up around half the volatility. This affects OTM options the most, but we are talking about options that are about to expire in a matter of hours, so there is a trade off between mispricing amount and whether this thing has any quotes left on it.

Usually just slightly out of the money is optimal for this liquidity vs mispricing trade-off. Keep in mind the ask will be 2x the price of the bid on a lot of these options.

A quick Monte Carlo

Running a quick Monte Carlo model, we can see that various discounts are revealed:

This is the output of 2 different models I developed to price these options, both based around Monte Carlos.

I made them both intentionally conservative. If it says 75% discount, it’s probably about twice as bad as that, but I didn’t want to be doing it wrong in the other direction.

I factored in the incredibly high volatility risk premium behind these options that are high gamma near expiry as well so you would likely collect this edge, although this comes with tails so not sure I would consider that a standalone strategy.

The Caveat

Sadly, there is a caveat here since you need to be willing to short a very high gamma option (most of these are overpriced since they are trading at 2x their implied vol as they should be). This is awful on your margin so you can only get so much efficiency.

The options are also fairly illiquid, it’s the type of alpha where you set up a pricing script with a telegram bot then manually pick off the market makers pricing these things.

You are still selling a nasty tail so not many are interested in that. I can’t say this is an immense alpha (if it was more actionable this would probably be a paid article), but it’s an interesting example about how market makers on Deribit either can’t be bothered or don’t think to price in this TWAP settlement into the volatility surface.

It’s a rite of passage in the quant world to fuck up your first backtest. I remember a particularly good backtest early on in my career that promptly sent me to the Lamborghini dealership website to size up which model I would soon be buying. Spoiler! This strategy did not buy me a Lamborghini.

In this specific case, I had messed up the timestamps on my data collection so data from the future was randomly stitched with current data, creating some too good to be true mean-reversion. It looked a little bit like this:

Which was really just two series (with the gaps filled in using red/green) that had stitched together poorly in my data scraper:

That was a fun lesson, and one I would learn many more times after. If your reaction upon seeing an amazing backtest isn’t “ah man what broke” then you haven’t seen enough of them to know better. There is no exception to this. Every entirely straight line I have ever generated in backtest has had some flaw, and the only super straight lines that actually realized in production weren’t backtest-able in the first place (market making). I’ve certainly had backtests that looked good and turned out to be good, but they never looked so good it was unbelievable - I certainly didn’t google any Lambos.

In the article today, I am going to dive into 20 different ways you can ruin your backtest. This isn’t so much meant to be a Wikipedia article on what overfitting is, since I’m sure you can find that easily, but more an example of many issues and caveats I’ve had to deal with throughout my career so that you can also be aware of them too.

Index

1. Lookahead

2. Overfitting

3. Survivorship Bias

4. Fees

5. Spread

6. Market Impact

7. Latency Assumptions

8. Limit Order Assumptions

9. Adversity Assumptions

10. Short Borrow

11. Funding Rates

12. Withdrawal Issues

13. Broken API

14. Assumed Infinite / Free Leverage

15. Tick Size Issues

16. Ignoring Gaming Dynamics

17. Assumed Trading Price is Trade-Able

18. Trade Price Bid/Ask Bounce

19. OTC Trades on Main Feed

20. Wash Flow
    

Lookahead

This is by far the most common cause of bad backtests. I’ll walk through some key ways that this can happen:

Some smoothing algorithms like Savitzky–Golay actually have lookahead inherent to their logic so whenever you are using a niche smoothing method, just make sure of this - it’s a mistake I’ve made before and it’s cost me a load of time figuring out what happened:

Another common way to introduce lookahead bias is when resampling. We will look below at the default version of pd.DataFrame.resample() compared to the correct way to do it:

If you use resample with the default arguments (Default: label=’left’, closed=’left’) then you get the example shown above which contains lookahead. We see that the high of 105 occurs at 04 but the timestamp is 05 — thus we have lookahead. When doing resample, always pass in the arguments label=’right’ and closed=’right’. Otherwise your timestamps will be at the open of the bar, which can easily lead to lookahead bias occurring, especially after merging dataframes.

There are about a trillion different ways that lookahead bias can occur and it usually produces a very strong effect in your data, so it’s the most likely answer when you have a really good PnL curve. The only other error that will make a curve look insanely good (completely straight line) is execution based assumptions that are wildly off such as limit fills + rebate + zero adversity + instant fill.

Overfitting

Overfitting is also fairly common. People like to tweak parameters until they eventually find something and a lot of the advice related to avoiding overfitting is fairly sensible. Always keep some data spare that you haven’t tested on. Whether that’s some newer data, a load of other assets, ideally both even, and then right at the end you can validate on this and if it fails you’ve used it and you call it a day. You need some piece of data ideally that you don’t play around with until the end.

Slowly increasing the amount of data you use until you are done the analysis helps. Visually, inspecting the backtest helps a lot as well. You can tell by how smooth the PnL curve is and how many trades were taken whether a curve has a high level of robustness. If all the PnL was made in 3 large jumps and it was flat otherwise then that means we have 3 events that made all our PnL. 3 isn’t many… You can have lots of trades and very few events that make the money, which can still happen in working strategies, positive skewness is a thing afterall, but it means you need to down-weight your mental view of how confident you are in this strategies ability to perform OOS (out of sample).

Double check out of sample very aggressively with not just one but 2 out of sample sets (validation set) when working with machine learning models because even after you fit it, you’ll do hyperparameter tuning.

On the subject of parameter tuning, if you make small-ish changes in your parameters and the overall trend of the curve flips completely then your alpha isn’t likely to be robust. You may see performance change a bit, but for great alphas the signal will shine through regardless of exact parameter choices. I.e. you can be very dirty on the machine learning, portfolio construction, and parameterization components and it’ll still make money.

Survivorship Bias

This is one that is not always worth preventing. Yes, you can use a dynamic universe, but guess what? That’s complicated and a real pain in the ass to code up. At some point you have to say — maybe I’ll only fix this one where it makes sense. If you have a fancy backtester then this makes sense to have in there, but if it’s a one off simulation, then I’m not sure this one always is worth the extra effort.

If you are doing long only momentum, best believe you will be bringing out that dynamic trading universe because survivorship bias will genuinely ruin your results.

If you are on the other hand doing a super delta neutral strategy or one that is directional but with equal long to short exposure over it’s history (which is much easier to check for than coding up a dynamic universe) then it’s worth skipping. I have very rarely had issues with survivorship bias.

It’s not an extremely strong force either so if you are working with 2 years of data and trying to find 3 Sharpe strategies, it isn’t going to be the thing ruining your backtest.

If you are doing 40 year backtests for heavily long strategies then this can destroy you backtest and rip the results to bits. It entirely depends on what you are simulating and whether we are doing it on long enough time horizons where it can make up a very large part of the total PnL at the end.

General tips here:

• If you are direction neutral you can skip it

• Affects longer term strategies more

Read more

This is not a modelling task exclusive to market makers, you also can run it as a taker. If a large order hits the market and you feel market makers have priced it poorly, you can pick them off. You don’t necessarily need to be fighting over microseconds, many impact models regard the evolution of prices over 15 minutes etc after the fill (mostly for large impacts only). That said, most impact models price in an average sized trade, and with a very short forecast horizon.

We’ll be working with the Wing model because SSVI is too simple for us, and I want to model put and call wing dynamics and not just simple volatility goes up vs down type of things.

Index

1. Introduction

2. Index

3. Data

4. Impact Modelling (Pre-Processing / Data Wrangling)

5. Impact Modelling

Data

In the previous article, we already scraped the data so you can either refer back to that or just read the same code below:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import sys
import os

tardis_api_key='get your own from tardis.dev (I don't get paid I just like them)'

from research_tools.scraping.tardis import *
from research_tools.preprocessing.process_oa_orders import parse_option_characteristics

exchange='deribit'

scrape_tardis_data(
    data_types_list=['quotes', 'trades', 'derivative_ticker'],
    start_date='2024-11-28',
    end_date='2024-12-02',
    instrument_type='futures',
    exchange_names=['deribit'],
    tardis_api_key=tardis_api_key,
    symbols_to_scrape=['BTC-PERPETUAL', 'ETH-PERPETUAL']
)

options_tickers = get_tickers(exchange, return_type='pandas')

options_tickers = options_tickers[options_tickers['type'] == 'option']
options_tickers = options_tickers[options_tickers['id'].str[:3].isin(["BTC"])]

options_tickers[['base_asset', 'expiry_dt', 'strike_price', 'option_type']] = options_tickers['id'].apply(
    lambda symbol: pd.Series(parse_option_characteristics(symbol, 'DERIBIT'))
)

expiry_dt = '2024-12-01 08:00:00'

options_tickers = options_tickers[options_tickers.expiry_dt == expiry_dt]
options_tickers = options_tickers[options_tickers.availableSince == "2024-11-28 00:00:00+00:00"]

scrape_tardis_data(
    data_types_list=['quotes', 'derivative_ticker', 'trades'],
    start_date='2024-11-28',
    end_date='2024-12-02',
    instrument_type='options',
    exchange_names=['deribit'],
    tardis_api_key=tardis_api_key,
    symbols_to_scrape=options_tickers['id'].tolist()
)

We will be working with quote data (to fit our curve), trade data (for our impacts), and derivative_ticker data (to adjust coin terms into USD terms).

Impact Modelling (Pre-Processing / Data Wrangling)

We will assume we are working in the same notebook as the first article, so please follow along with the code there to understand how to get a DataFrame of parameters as we will be analysing this.

params = {}
for ts in tqdm(timestamps):
    current_index_price = index_px.loc[ts].index_price
    otm_options = get_otm_options(current_index_price, symbols)
    current_df = get_quotes_for_timestamp(options_data, otm_options, ts)
    current_df = adjust_prices(current_index_price, current_df)
    prepped_wing_df = prepare_data_for_wing_model(current_df, expiry_dt, current_index_price)
    x_array   = prepped_wing_df["x"].values
    iv_array  = prepped_wing_df["iv"].values
    vega_array= prepped_wing_df["vega"].values
    p, loss, arb_indicator = ArbitrageFreeWingModel.calibrate(
        x_array, iv_array, vega_array,
        dc=-0.2,
        uc=0.2,
        dsm=0.5,
        usm=0.5,
        epochs=10,
        use_constraints=True
    )
    params[ts] = p

As a note related to our params, you can generate it for the Wing model instead, with this as an example for our regular interval parameters (we will use a slightly different timestamping). The only reason I have stuck with SSVI is because it’s much slower to use the Wing model. Ideally, we would have all of this in some Rust or C++ tooling with Python bindings, but instead we are stuck with slow Python for now. In a later article, we may convert them… we’ll see how far this series goes.

This time around, we need trade data, so let’s load that in:

symbols = options_tickers['id'].tolist()

trade_data = {}

for symbol in symbols:
    data_folder_path = Path(os.path.join(base_folder_path, symbol, 'trades'))
    gz_files = list(data_folder_path.glob('*.csv.gz'))
    curr_df = pd.DataFrame()
    for gz_file in gz_files:
        try:
            temp_df = pd.read_csv(gz_file, compression='gzip')
            if not temp_df.empty:
                temp_df['timestamp'] = pd.to_datetime(temp_df['timestamp'], unit='us')
                temp_df = temp_df.set_index('timestamp', drop=True)
                curr_df = pd.concat([curr_df, temp_df], axis=0).sort_index()
        except:
            pass
    trade_data[symbol] = curr_df

If you haven’t noticed, it’s the same code as we used for loading in the options quote data, but with a renaming of the dict and changing the folder to trades (this is specific to how I have my flat files stored, change for your own).

df_trades = pd.DataFrame()
for symbol, df in trade_data.items():
    df_trades = pd.concat([df_trades, df], axis=0)

We’ll aggregate this back up again, probably could have done that in the earlier code, but we were re-using old code so this is easier…

df_trades = df_trades.sort_index()

Sort the index as well, we want the trades in order. Next, we will shift the index, you are welcome to play around with how much we shift it, but it’s effectively our lookahead horizon. Keep in mind that fitting a vol curve is expensive computationally so in crypto the prices tend not to update insanely fast like they would in perps. 2.5 seconds is much more than it takes, but I figured I would be safe:

lookahead_time_usec = 2_500_000
forward_shifted_index = list(df_trades.index + pd.Timedelta(microseconds=lookahead_time_usec))
unshifted_index = list(df_trades.index)

Now, let’s get our before, and after datasets:

import py_vollib_vectorized

before_params = {}
trade_data = {}
for i, ts in tqdm(enumerate(unshifted_index), total=len(unshifted_index)):
    current_trade = df_trades.iloc[i]
    pos = index_px.index.searchsorted(ts, side='left')
    current_index_price = index_px.iloc[pos].index_price
    current_trade['price'] *= current_index_price
    otm_options = get_otm_options(current_index_price, symbols)
    current_df = get_quotes_for_timestamp(options_data, otm_options, ts)
    p = None
    delta = None
    if not current_df.empty:
        current_df = adjust_prices(current_index_price, current_df)
        p, curve, mkt_ivs = fit_ssvi_from_df(current_df, expiry_dt, current_index_price)
        base_asset, expiry_dt, strike_px, option_type = parse_option_characteristics(current_trade['symbol'], current_trade['exchange'])
        iv = curve(np.log(strike_px / current_index_price))
        t = (expiry_dt - current_trade.name).total_seconds() / 31_536_000
        r = 0
        delta = py_vollib_vectorized.vectorized_delta(
            option_type.lower(),
            current_index_price,
            strike_px,
            t, 
            r, 
            iv,
            model='black_scholes',
            return_as='numpy'
        )[0]
    trade_data[ts] = {"delta" : delta, "size": current_trade["amount"], "side": current_trade['side'], "price": current_trade['price']} 
    before_params[ts] = p

after_params = {}
for ts in tqdm(forward_shifted_index):
    pos = index_px.index.searchsorted(ts, side='left')
    current_index_price = index_px.iloc[pos].index_price
    otm_options = get_otm_options(current_index_price, symbols)
    current_df = get_quotes_for_timestamp(options_data, otm_options, ts)
    p = None
    if not current_df.empty:
        current_df = adjust_prices(current_index_price, current_df)
        p, curve, mkt_ivs = fit_ssvi_from_df(current_df, expiry_dt, current_index_price)
    after_params[ts] = p

And put them in a DataFrame:

df_before_params = pd.DataFrame(before_params)
df_before_params = df_before_params.T
df_before_params

df_after_params = pd.DataFrame(after_params)
df_after_params = df_after_params.T
df_after_params

df_trade_data = pd.DataFrame(trade_data)
df_trade_data = df_trade_data.T
df_trade_data

And to merge them we unshift the index on the after frame:

df_after_params.index -= pd.Timedelta(microseconds=lookahead_time_usec)

Then we can merge them:

df_params = pd.DataFrame(index=df_before_params.index)

for col_name in df_before_params:
    df_params[f'{col_name}_before'] = df_before_params[col_name]
    df_params[f'{col_name}_after'] = df_after_params[col_name]

for col_name in df_trade_data:
    df_params[col_name] = df_trade_data[col_name]

df_params = df_params.dropna()

Let's add a few extra columns I think will be useful:

Read more

It’s been a busy few months for me, as I’ve started a new role a few months back in the world of linear instruments I’ve decided to write an article regarding some of my previous experience in options and how fair values are fitted as well as profited from.

When it comes down to it, the core money maker for making and taking remains roughly the same. Sure, you have maker models which don’t rely on vol curves (although they’re extremely simple to say the least) and there’s taker strategies that don’t do this, but by and large this is the most popular way of trading options at large institutions.

For a rough overview of how this article will be structured, we will first start with explaining how to fit some basic SSVI and SVI models. Then we’ll move on to working with the Wing model. Then we’ll explore a taker strategy where we exploit differences between the market fit vol curve and our values. We will also talk about how market makers fit their parameters that they then end up quoting around. In general, this is an options market making 101 guide for the pricing side of things.

This article will work with 2 models. The Orc Wing model which was a model used by Optiver as well as a range of other firms well over a decade ago, as well as SSVI (Stochastic Skew Volatility Inspired) which is publicly available in the literature. In modern times, Vola Dynamics is the choice model for most firms, if not an in-house proprietary solution. I’ve used Vola before but I can’t afford the $25,000-30,000 price tag per month or at least not until many many more people subscribe to this blog just for personal use and I’d have been fired if I used it to write an article at work so we’ll stick to slightly behind the leading edge.

We will be fitting cryptocurrency markets today which probably could use a C9 (9-parameter) curve as it isn’t exactly super advanced and there aren’t things like VIX options which can actually get you picked off on the tails if you were quoting something like SPY options (in which case you would be in the high teens, maybe even 20s).

Above is what one of these looks like. Sadly, C9 curves are from Vola Dynamics so we will be using a 6 parameter model with the Orc model and an even more simplified model with only 3 parameters under the SSVI model. The reason I bring it up of course is to emphasize that unlike equity options, the markets for cryptocurrencies don’t require as complex a model and your time is best spent on figuring out other types of dynamics / strategy optimizations outside of the vol curve - which I won’t delve into now. SSVI is probably a bit too simple, but we’ll get away with it.

In all honesty, even if you were quoting around the Orc Wing model you’d probably be able to do quite well if your parameters had been fit properly (although if you have the opex to spend, get Vola still) in cryptocurrency markets whereas I would reserve the SSVI model for taking only whereas Orc is more capable of both. In equities, you’ll need Vola but that's a whole new ballpark

Index

1. Introduction

2. What’s a Vol Curve?

3. How to fit an SVI model so that it is arbitrage-free

4. Options Data Wrangling

5. Fitting SSVI parameters to the market

6. Patterns in SSVI models and exploiting them

7. When SSVI isn’t enough (it rarely is enough)

8. How to trade parameters themselves

   1. Maker

   2. Taker

9. Appendix: Orc Wing Model Code

What’s a Vol Curve?

Starting with a more proper explanation, a vol curve, like an SSVI curve, is basically a smooth function that maps the strike prices (or moneyness) of options to their implied volatilities in a way that looks realistic and consistent with market data. Think of it this way: for different options on the same underlying asset but with different strike prices, the implied volatility can vary—forming what’s called a “smile” or “skew.” An SSVI curve is one mathematically elegant method to fit (or interpolate) these implied volatility smiles so that they move in a plausible way over time and across strikes, so that traders (or risk managers, really whoever is using it at these prop firms) to price options consistently.

Really, a volatility curve is a generalization across all of the different options prices so we can reduce the dimensionality of what is a very high dimensionality asset class.

If I get filled on an option with X delta, Y gamma, etc etc, do I really want to have to fit a parameter for every single option I am quoting as to how this should change my pricing? And even if I do have that what if the delta or gamma of that option changes - how do those parameters even change? The point is, you just can’t do it the same way as it’s done in equities where you can simply have a pricing model for each different symbol and that’s it. Whether that’s the optimal model is a different question, but importantly - it can be done.

So what are these parameters and what do they represent for each of our models?

Let’s start first with the simpler model, SSVI. We have 3 main parameters: theta, rho, and phi. (k is just the strike in the below formula)

θ - Theta represents the total implied variance at-the-money. It’s basically the total level of the curve and doesn’t affect the shape of the curve, merely how high it is overall. Here’s what happens when we play around with it theta:

ρ - Rho represents the skew or the asymmetry of the curve. If we rotate the curve around we are saying put volatility is more expensive than calls or vice versa. It let’s us express whether we think large up or down moves are more likely. If we vary it we get this:

ϕ - Phi is the convexity of the curve, it changes how likely we think extreme moves are. Here’s what variations look like:

For those following along at home, here’s the code to run:

import numpy as np
import matplotlib.pyplot as plt

def ssvi_total_variance(k: float, theta: float, rho: float, phi: float) -> float:
    """
    Returns total implied variance w(k) under the SSVI parameterization:
    w(k) = (theta/2) * [ 1 + rho * phi * k + sqrt((phi*k + rho)**2 + 1 - rho**2) ]
    """
    return (theta / 2.0) * (
        1.0 
        + rho * phi * k 
        + np.sqrt((phi * k + rho) ** 2 + 1 - rho**2)
    )

# Create a range of log-moneyness points k
k_values = np.linspace(-2, 2, 200)

# -------------------------------
# 1) Vary theta
# -------------------------------
theta_list = [0.1, 0.3, 0.5, 0.7, 0.9]
rho_fixed = 0
phi_fixed = 2.00

plt.figure(figsize=(15, 4))

plt.subplot(1, 3, 1)
for theta in theta_list:
    w = ssvi_total_variance(k_values, theta, rho_fixed, phi_fixed)
    implied_vol = np.sqrt(w)  # T = 1 for simplicity
    plt.plot(k_values, implied_vol, label=f"theta={theta:.1f}")
plt.title("Effect of varying theta")
plt.xlabel("k (log-moneyness)")
plt.ylabel("Implied Vol")
plt.legend()

# -------------------------------
# 2) Vary rho
# -------------------------------
rho_list = [-0.9, -0.5, 0.0, 0.5, 0.9]
theta_fixed = 0.5
phi_fixed = 1.0

plt.subplot(1, 3, 2)
for rho in rho_list:
    w = ssvi_total_variance(k_values, theta_fixed, rho, phi_fixed)
    implied_vol = np.sqrt(w)
    plt.plot(k_values, implied_vol, label=f"rho={rho:.1f}")
plt.title("Effect of varying rho")
plt.xlabel("k (log-moneyness)")
plt.ylabel("Implied Vol")
plt.legend()

# -------------------------------
# 3) Vary phi
# -------------------------------
phi_list = [0.5, 1.0, 1.5, 2.0, 2.5]
theta_fixed = 0.5
rho_fixed = 0

plt.subplot(1, 3, 3)
for phi in phi_list:
    w = ssvi_total_variance(k_values, theta_fixed, rho_fixed, phi)
    implied_vol = np.sqrt(w)
    plt.plot(k_values, implied_vol, label=f"phi={phi:.1f}")
plt.title("Effect of varying phi")
plt.xlabel("k (log-moneyness)")
plt.ylabel("Implied Vol")
plt.legend()

plt.tight_layout()
plt.show()

How to fit an SVI model so that it is arbitrage-free

The formula I showed earlier is SSVI, it is the re-parameterization of SVI that enforces certain smoothness or arbitrage-free conditions more naturally (especially when done across expiries). In the below article, I showed some code for doing this with the raw SVI model which is generally a simpler model:

I won’t delve into the raw SVI model here since all the code and plots are in this article, but I will show how we fit the SSVI model in this chapter. There is also code for the Orc Wing model in that article, which I will actually toy about with in this article (later chapter).

Let’s first start with a slightly more complicated approach to things before leaving the SVI world for the SSVI and eventually Wing world. We will enforce no-arbitrage constraints using a penalty-based approach. We will enforce both butterfly and calendar arbitrage bounds for our options:

Let’s explain what these terms mean. First of all we have { a, b, σ, ρ, m }. These are rather simple and are the 5 parameters for the original SVI model (which can be read more about in the article mentioned earlier).

Next, we have w_raw​( x; a, b, σ, ρ, m ) - this denotes the total implied variance at log-moneyness x. It’s basically just the model’s prediction for how much variance an option at strike x should have.

w_mkt is the market observed implied variance - our observed data.

∑n i=1​(…)^2 - This is the least-squares term for optimization.

𝜏_i represents the time to expiry for the i-th maturity.

λ_cal represents a penalty weight for calendar arbitrage. If the model violates the arbitrage bound that variance should not decrease as time-to-expiration increases, a penalty is applied. The larger this value the more strictly we are enforcing the no-arbitrage bound.

​max(0, w(x, 𝜏_i − 1​) − w(x, 𝜏_i​))^2 - this captures the calendar arbitrage violations. For a later maturity compared to an earlier maturity we want w(x, 𝜏_i − 1​) <= w(x, 𝜏_i​).

λ_fly represents the penalty weight for butterfly arbitrage similarly to how we set up the penalty for calendar arbitrage. It ensures the implied volatility curve remains convex enough to avoid negatively priced butterfly spreads. A larger value for this means a more strict enforcement of convexity.

Finally, g(x;a,b,σ,ρ,m) is out function which checks for butterfly arbitrage. If the SVI parameters lead to concavity or other violations, g becomes negative, and you pay a penalty max(0,−g(⋅))^2.

Okay… That’s enough math. Let’s put this into code, here’s our penalty functions:

Read more

Read more