version 0.12.0

Author: Robert Carver

DONE_TO_DO.md

@@ -1,5 +1,10 @@
 # Release notes
 
+## Version 0.12.0
+
+* Capital correction now works. New methods: system.accounts.capital_multiplier, system.accounts.portfolio_with_multiplier, system.portfolio.get_actual_positon, system.portfolio.get_actual_buffers_with_position, system.accounts.get_buffered_position_with_multiplier. See this [blog post](http://qoppac.blogspot.co.uk/2016/06/capital-correction-pysystemtrade.html)  and [the guide](https://github.com/robcarver17/pysystemtrade/blob/master/docs/userguide.md#capcorrection)
+
+
 ## Version 0.11.2
 
 * Smooth fixed weights as well as variable: removed ewma_span and moved to new config item forecast_weight_ewma_span and same for instruments. Removed override of get_instrument_weights, get_forecast_weights method from estimated classes.

README.md

@@ -7,10 +7,10 @@ Rob Carver
 [http://qoppac.blogspot.co.uk/p/pysystemtrade.html](http://qoppac.blogspot.co.uk/p/pysystemtrade.html)
 
 
-Version 0.11.1
+Version 0.12.0
 
 
-20160528
+20160608
 
 
 ## Release notes

docs/userguide.md

@@ -2725,9 +2725,6 @@ base_currency: "USD"
 Note that the stage code tries to get the percentage volatility of an instrument from the rawdata stage. Since a rawdata stage might be omitted, it can also fall back to calculating this from scratch using the data object and the default volatility calculation method.
 
 
-#### New or modified position scaling stages
-
-In the future I'll update this stage to allow for variable capital.
 
 <a name="stage_portfolio">
 ### Stage: Creating portfolios [Portfolios class](/systems/portfolio.py)
@@ -2963,7 +2960,7 @@ acc_curve.costs.weekly.percent().curve()
 ```
 
 
-You may also want to use *cumulated* returns, which use compound interest rather than the simple addition I normally use. See this blog for more information FIX ME
+You may also want to use *cumulated* returns, which use compound interest rather than the simple addition I normally use. See this [blog post](http://qoppac.blogspot.co.uk/2016/06/capital-correction-pysystemtrade.html).
 
 ```python
 acc_curve.cumulative()
@@ -3211,7 +3208,7 @@ I plan to include ways of summarising profits over groups of assets (trading rul
 # Processes
 </a>
 
-This section gives much more detail on certain important processes that span multiple stages: logging, estimating correlations and diversification multipliers, and optimisation.
+This section gives much more detail on certain important processes that span multiple stages: logging, estimating correlations and diversification multipliers, optimisation, and capital correction.
 
 <a name="logging">
 ## Logging
@@ -3488,6 +3485,36 @@ instrument_div_mult_estimate:
 I've included a smoothing function, other wise jumps in the multiplier will cause trading in the backtest. Note that the FDM is calculated on an instrument by instrument basis, but if instruments have had their forecast weights and correlations estimated on a pooled basis they'll have the same FDM. It's also a good idea to floor negative correlations at zero to avoid inflation the DM to very high values.
 
 
+<a name="capcorrection">
+## Capital correction: Varying capital
+</a>
+
+Capital correction is the process by which we change the capital we have at risk, and thus our positions, according to any profits or losses made. Most of pysystemtrade assumes that capital is *fixed*. This has the advantage that risk is stable over time, and account curves can more easily be interpreted. However a more common method is to use *compounded* capital, where profits are added to capital and losses deducted. If we make money then our capital, and the risk we're taking, and the size of our positions, will all increase over time.
+
+There is much more in this [blog post](http://qoppac.blogspot.co.uk/2016/06/capital-correction-pysystemtrade.html). Capital correction is controlled by the following config parameter which selects the function used for correction using the normal dot argument (the default here being the function `fixed_capital` in the module `syscore.capital`)
+
+YAML:
+``` 
+capital_multiplier:
+   func: syscore.capital.fixed_capital
+```
+
+Other functions I've written are `full_compounding` and `half_compounding`. Again see the blog post [blog post](http://qoppac.blogspot.co.uk/2016/06/capital-correction-pysystemtrade.html) for more detail.
+
+To get the varying capital multiplier which the chosen method calculates use `system.accounts.capital_multiplier()`. The multiplier will be 1.0 at a given time if the variable capital is identical to the fixed capital.
+
+Here's a list of methods with their counterparts for both fixed and variable capital:
+
+|                             | Fixed capital | Variable capital       | 
+|:-------------------------:|:---------:|:---------------:|
+| Get capital at risk | `positionSize.get_daily_cash_vol_target()['notional_trading_capital']`  |  `accounts.get_actual_capital()` |  
+| Get position in a system portfolio | `portfolio.get_notional_position` | `portfolio.get_actual_position` |
+| Get buffers for a position | `portfolio.get_buffers_for_position` | `portfolio.get_actual_buffers_for_position` |
+| Get buffered position | `accounts.get_buffered_position`| `accounts.get_buffered_position_with_multiplier`| 
+| Get p&l for instrument at system level |  `accounts.pandl_for_instrument`|  `accounts.pandl_for_instrument_with_multiplier`|
+| P&L for whole system |  `accounts.portfolio`|  `accounts.portfolio_with_multiplier`| 
+
+All other methods in pysystemtrade use fixed capital.
 
 <a name="reference">
 # Reference
@@ -3646,6 +3673,8 @@ Other methods exist to access logging and cacheing.
 | `portfolio.get_instrument_diversification_multiplier`| Standard / Estimate |  | D |Get instrument div. multiplier |
 | `portfolio.get_notional_position`| Standard | `instrument_code` | D,O |Get the *notional* position (with constant risk capital; doesn't allow for adjustments when profits or losses are made) |
 | `portfolio.get_buffers_for_position`| Standard | `instrument_code` | D,O |Get the buffers around the position |
+| `portfolio.get_actual_position`| Standard |  `instrument_code` | D,O | Get position accounting for capital multiplier|
+| `portfolio.get_actual_buffers_for_position`| Standard | `instrument_code` | D,O |Get the buffers around the position, accounting for capital multiplier |
 
 
 
@@ -3656,6 +3685,7 @@ Inputs:
 | Call                              | Standard?| Arguments       | Type | Description                                                    |
 |:-------------------------:|:---------:|:---------------:|:----:|:--------------------------------------------------------------:|
 | `accounts.get_notional_position`| Standard |  `instrument_code` | I | `portfolio.get_notional_position`|
+| `accounts.get_actual_position`| Standard |  `instrument_code` | I | `portfolio.get_actual_position`|
 | `accounts.get_capped_forecast`| Standard | `instrument_code`, `rule_variation_name`  | I | `forecastScaleCap.get_capped_forecast`|
 | `accounts.get_instrument_list`| Standard |  | I | `portfolio.get_instrument_list`|
 | `accounts.get_notional_capital`| Standard |  | I | `positionSize.get_daily_cash_vol_target`|
@@ -3665,6 +3695,7 @@ Inputs:
 | `accounts.get_daily_returns_volatility`| Standard |  `instrument_code` | I | `rawdata.daily_returns_volatility` or `data.daily_prices`|
 | `accounts.get_raw_cost_data`| Standard | `instrument_code` | I  | `data.get_raw_cost_data` |
 | `accounts.get_buffers_for_position`| Standard |  `instrument_code` | I | `portfolio.get_buffers_for_position`|
+| `accounts.get_actual_buffers_for_position`| Standard |  `instrument_code` | I | `portfolio.get_actual_buffers_for_position`|
 | `accounts.get_instrument_diversification_multiplier`| Standard |   | I | `portfolio.get_instrument_diversification_multiplier`|
 | `accounts.get_instrument_weights`| Standard |   | I | `portfolio.get_instrument_weights`|
 | `accounts.get_trading_rules_list`| Standard | `instrument_code`  | I | `combForecast.get_trading_rule_list`|
@@ -3681,15 +3712,21 @@ Diagnostics:
 | `accounts.get_instrument_forecast_scaling_factor`| Standard | `instrument_code`, `rule_variation_name`  | D | IDM * instrument weight * FDM * forecast weight|
 | `accounts.get_capital_in_rule`| Standard |  `rule_variation_name`  | D | Sum of `get_instrument_forecast_scaling_factor` for a given trading rule|
 | `accounts.get_buffered_position`| Standard |  `instrument_code` | D | Buffered position at portfolio level|
+| `accounts.get_buffered_position_with_multiplier`| Standard |  `instrument_code` | D | Buffered position at portfolio level, including capital multiplier|
 | `accounts.subsystem_turnover`| Standard | `instrument_code`  | D | Annualised turnover of subsystem|
 | `accounts.instrument_turnover`| Standard | `instrument_code`  | D | Annualised turnover of instrument position at portfolio level|
 | `accounts.forecast_turnover`| Standard | `instrument_code`, `rule_variation_name`  | D | Annualised turnover of forecast|
 | `accounts.get_SR_cost_for_instrument_forecast`| Standard | `instrument_code`, `rule_variation_name`  | D | SR cost * turnover for forecast|
+| `accounts.capital_multiplier`| Standard |   | D, O | Capital multiplier, ratio of actual to fixed notional capital|
+| `accounts.get_actual_capital`| Standard |  | D | Actual capital (fixed notional capital times multiplier)|
 
 
 Accounting outputs:
 
+| Call                              | Standard?| Arguments       | Type | Description                                                    |
+|:-------------------------:|:---------:|:---------------:|:----:|:--------------------------------------------------------------:|
 | `accounts.pandl_for_instrument`| Standard |  `instrument_code` | D | P&l for an instrument within a system|
+| `accounts.pandl_for_instrument_with_multiplier`| Standard |  `instrument_code` | D | P&l for an instrument within a system, using multiplied capital|
 | `accounts.pandl_for_instrument_forecast`| Standard | `instrument_code`, `rule_variation_name` | D | P&l for a trading rule and instrument |
 | `accounts.pandl_for_instrument_forecast_weighted`| Standard | `instrument_code`, `rule_variation_name` | D | P&l for a trading rule and instrument as a % of total capital |
 | `accounts.pandl_for_instrument_rules`| Standard | `instrument_code` | D,O | P&l for all trading rules in an instrument, weighted |
@@ -3702,6 +3739,7 @@ Accounting outputs:
 | `accounts.pandl_for_all_trading_rules`| Standard |  | D | P&l for trading rules across whole system |
 | `accounts.pandl_for_all_trading_rules_unweighted`| Standard |  | D | P&l for trading rules across whole system |
 | `accounts.portfolio`| Standard |  | O,D | P&l for whole system |
+| `accounts.portfolio_with_multiplier`| Standard |  | D | P&l for whole system using multiplied capital|
 
 
 
@@ -4424,3 +4462,15 @@ forecast_cost_estimate:
    use_pooled_turnover: True ### Use weighted average of turnover across instruments with the same set of trading rules
 ```
 
+#### Capital correction
+
+Which capital correction method should we use?
+
+YAML:
+```
+capital_multiplier:
+   func: syscore.capital.fixed_capital
+```
+
+Other valid functions include full_compounding and half_compounding.
+

examples/breakout/temp.py

@@ -1,17 +0,0 @@
-from matplotlib.pyplot import show, title
-from systems.provided.futures_chapter15.estimatedsystem import futures_system
-system = futures_system(log_level="on")
-system.config.capital_multiplier['func']='syscore.capital.full_compounding'
-
-system.accounts.capital_multiplier().plot()
-show()
-
-system.accounts.get_actual_capital().plot()
-show()
-
-
-system.accounts.portfolio_with_multiplier().percent().curve().plot()
-show()
-
-system.accounts.portfolio().percent().curve().plot()
-show()

examples/variablecapital/variablecapital.py

@@ -0,0 +1,72 @@
+from matplotlib.pyplot import show
+from systems.provided.futures_chapter15.basesystem import futures_system
+
+system = futures_system(log_level="on")
+system.config.instrument_weights=dict(EDOLLAR=1.0)
+
+system.config.capital_multiplier['func']='syscore.capital.fixed_capital'
+"""
+system.accounts.portfolio().curve().plot()
+show()
+
+
+
+system.accounts.portfolio().percent().curve().plot()
+show()
+
+system.accounts.portfolio().cumulative().curve().plot()
+show()
+"""
+pandl_fixed=system.accounts.portfolio()
+
+print(system.accounts.portfolio().capital)
+
+"""
+system = futures_system(log_level="on")
+system.config.instrument_weights=dict(EDOLLAR=1.0)
+system.config.capital_multiplier['func']='syscore.capital.full_compounding'
+
+system.accounts.capital_multiplier().plot()
+show()
+
+system.accounts.portfolio_with_multiplier().capital.plot()
+show()
+
+system.accounts.portfolio_with_multiplier().curve().plot()
+show()
+
+
+
+system.accounts.get_buffered_position_with_multiplier("EDOLLAR", False).plot()
+system.accounts.get_buffered_position("EDOLLAR", False).plot()
+show()
+
+
+"""
+
+system = futures_system(log_level="on")
+system.config.instrument_weights=dict(EDOLLAR=1.0)
+system.config.capital_multiplier['func']='syscore.capital.half_compounding'
+
+"""
+system.accounts.capital_multiplier().plot()
+show()
+
+system.accounts.portfolio_with_multiplier().capital.plot()
+show()
+
+system.accounts.portfolio_with_multiplier().curve().plot()
+show()
+
+
+
+system.accounts.get_buffered_position_with_multiplier("EDOLLAR", False).plot()
+system.accounts.get_buffered_position("EDOLLAR", False).plot()
+show()
+"""
+
+pandl_variable=system.accounts.portfolio_with_multiplier()
+pandl_fixed.curve().plot()
+pandl_variable.curve().plot()
+show()
+

syscore/accounting.py

@@ -352,12 +352,6 @@ def cumulative(self):
 
         return new_curve
 
-    def as_cum_percent(self):
-        perc_prod_returns = (self.as_percent()/100.0) + 1.0
-        
-        perc_ac_curve_cum = perc_prod_returns.cumprod() 
-        
-        return 100.0*perc_ac_curve_cum
 
 
     def as_percent(self):
@@ -371,11 +365,13 @@ def as_cumulative(self):
         else:
             use_capital=self.capital
 
-        perc_ac_curve_cum = self.as_cum_percent() / 100.0
+        perc_ac_returns = self.as_percent() / 100.0
+        
+        cum_returns = (1.0 + perc_ac_returns).cumprod()
 
-        cum_returns = perc_ac_curve_cum * use_capital
+        cum_returns = cum_returns * use_capital
 
-        return cum_returns
+        return cum_returns.diff()
 
 
     def curve(self):

syscore/capital.py

@@ -1,19 +1,35 @@
 """
 Functions to calculate capital multiplier
 
-Change to pass what is needed
+ALl return Tx1 pd.Series
 """
+from copy import copy
+import pandas as pd
+import numpy as np
 
 def fixed_capital(system, **ignored_args):
-    pandl  = system.accounts.portfolio().percent()
-    pandl[:]=1.0
+    multiplier  = copy(system.accounts.portfolio().percent())
+    multiplier[:]=1.0
 
-    return pandl
+    return multiplier
 
 def full_compounding(system, **ignored_args):
-    pandl  = system.accounts.portfolio().percent().curve()
-    pandl = pandl / 100.0
-    return pandl
+    pandl  = system.accounts.portfolio().percent()
+    multiplier = 1.0 + (pandl / 100.0)
+    multiplier = multiplier.cumprod().ffill()
+    
+    return multiplier
 
 def half_compounding(system, **ignored_args):
-    pass
+    pandl  = system.accounts.portfolio().percent().curve().ffill().diff()
+    multiplier=1.0
+    multiplier_list = []
+    for daily_return in pandl: 
+        actual_return = multiplier * daily_return / 100.0
+        multiplier = multiplier * (1.0 + actual_return)
+        multiplier = np.nanmin([multiplier, 1.0])
+        multiplier_list.append(multiplier)
+        
+    multiplier = pd.Series(multiplier_list, index=pandl.index).ffill()
+    
+    return multiplier

systems/account.py

@@ -378,6 +378,21 @@ def get_buffers_for_position(self, instrument_code):
 
         return self.parent.portfolio.get_buffers_for_position(instrument_code)
 
+    def get_actual_buffers_for_position(self, instrument_code):
+        """
+        Get the actual capital corrected buffered position from a previous module
+
+        :param instrument_code: instrument to get values for
+        :type instrument_code: str
+
+        :returns: Tx2 pd.DataFrame: columns top_pos, bot_pos
+
+        KEY INPUT
+        """
+        
+        return self.parent.portfolio.get_actual_buffers_for_position(instrument_code)
+
+
     def get_daily_price(self, instrument_code):
         """
         Get the instrument price from rawdata
@@ -1839,19 +1854,6 @@ def get_actual_position(self, instrument_code):
         return self.parent.portfolio.get_actual_position(instrument_code)
 
 
-    def get_actual_buffers_for_position(self, instrument_code):
-        """
-        Get the buffered position for actual positions from a previous module
-
-        :param instrument_code: instrument to get values for
-        :type instrument_code: str
-
-        :returns: Tx2 pd.DataFrame: columns top_pos, bot_pos
-
-        KEY INPUT
-        """
-        
-        return self.parent.portfolio.get_actual_buffers_for_position(instrument_code)
 
     def get_buffered_position_with_multiplier(self, instrument_code, roundpositions=True):
         """

systems/provided/defaults.yaml

@@ -104,6 +104,8 @@ forecast_weight_estimate:
 percentage_vol_target: 16.0
 notional_trading_capital: 1000000
 base_currency: "USD"
+capital_multiplier:
+   func: syscore.capital.fixed_capital
 #
 # Portfolio creation
 #
@@ -169,6 +171,4 @@ buffer_trade_to_edge: False
 # costs and accounting
 use_SR_costs: True
 #
-capital_multiplier:
-   func: syscore.capital.fixed_capital