usim

Dataloader: BaseData

Data loader for base data.

Sample config:

<module id="BaseData" type="data"
path="${usimpath}/bin/libdlbasedata.so"
datapath="/dat/all/Sample/Equity/Raw_Data/Equity/Data/${YYYY}/${MM}/${DD}/${YYYY}-${MM}-${DD}.txt"
needtag="false" hasheader="true" separator="|"/>

Dependency: none

parameter	type	default	description
datapath	string	required	the path of raw data
needtag	bool	false	true if need tag when load data
hasheader	bool	false	true if the first line of the raw data is header
separator	char	comma(,)	the char that separate data

datatag	type	description
open	Udata_Matrix $<$ float32 $>$	none
close	Udata_Matrix $<$ float32 $>$	none
close_prev	Udata_Matrix $<$ float32 $>$	none
high	Udata_Matrix $<$ float32 $>$	none
low	Udata_Matrix $<$ float32 $>$	none
tvrvolume	Udata_Matrix $<$ float32 $>$	none
tvrdeals	Udata_Matrix $<$ float32 $>$	none
sttype	Udata_Matrix $<$ int32 $>$	none
split	Udata_Matrix $<$ float32 $>$	none
dividend	Udata_Matrix $<$ float32 $>$	none
actualplaratio	Udata_Matrix $<$ float32 $>$	none
plaprice	Udata_Matrix $<$ float32 $>$	none
baseshares	Udata_Matrix $<$ float32 $>$	none
actualplavol	Udata_Matrix $<$ float32 $>$	none
totalshares	Udata_Matrix $<$ float32 $>$	none
totalfloatshares	Udata_Matrix $<$ float32 $>$	none
zjhsector	Udata_Matrix $<$ int32 $>$	none
zjhindustry	Udata_Matrix $<$ int32 $>$	none
swsector	Udata_Matrix $<$ int32 $>$	none
swindustry	Udata_Matrix $<$ int32 $>$	none
swsubindustry	Udata_Matrix $<$ int32 $>$	none
jysector	Udata_Matrix $<$ int32 $>$	none
jyindustry	Udata_Matrix $<$ int32 $>$	none
jysubindustry	Udata_Matrix $<$ int32 $>$	none

Dataloader: adjfactor

Data loader for the factor of adjusted prices.

Sample config:

<module id="adjfactor" type="data"
path="${usimpath}/bin/libdladjfactor.so" />

Dependency: BaseData

datatag	type	description
factor	Udata_Matrix $<$ float32 $>$	adj_close[di-1][ii] = close[di-1][ii] / factor[di][ii]. When split or dividend happens, factor > 1. otherwise factor=1

Dataloader: AdjPrices

Data loader for adjusted prices.

Sample config:

<module id="AdjPrices" type="data"
path="${usimpath}/bin/libdladjprices.so" needtag="false"/>

Dependency: BaseData, return

datatag	type	description
adj_open	Udata_Matrix $<$ float32 $>$	none
adj_high	Udata_Matrix $<$ float32 $>$	none
adj_low	Udata_Matrix $<$ float32 $>$	none
adj_tvrvolume	Udata_Matrix $<$ float32 $>$	none
adj_factor	Udata_Matrix $<$ float32 $>$	when split or dividend occurs, adj_factor $>$ 1

Dataloader: index

Data loader for index data.

Sample config:

<module id="index" type="data"
path="${usimpath}/bin/libdlindex.so"
datapath="/dat/all/Sample/Equity/Raw_Data/Index/Data/${YYYY}/${MM}/${DD}/${YYYY}-${MM}-${DD}.txt"
instruments="SH000300 SH000001 SZ399001"
hasheader="true" separator="|" />

Dependency: none

parameter	type	default	description
datapath	string	required	the path of raw data
needtag	bool	false	true if need tag when load data
hasheader	bool	false	true if the first line of the raw data is header
separator	char	comma(,)	the char that separate data
instruments	string	required	instruments' id. Separated with space

datatag	type	description
[tag].[inst]_close	Udata_Matrix $<$ float32 $>$	none
[tag].[inst]_open	Udata_Matrix $<$ float32 $>$	none
[tag].[inst]_high	Udata_Matrix $<$ float32 $>$	none
[tag].[inst]_low	Udata_Matrix $<$ float32 $>$	none
[tag].[inst]_tvrvolume	Udata_Matrix $<$ float32 $>$	none
[tag].[inst]_tvrvalue	Udata_Matrix $<$ float32 $>$	none
[tag].[inst]_tvrdeals	Udata_Matrix $<$ float32 $>$	none

note: if SH00300 is in parameter "instruments", then data includes [tag].SH00300_close, [tag].SH00300_open, [tag].SH00300_high, and so on.

Dataloader: suspend

Data loader for suspend data.

Sample config:

<module id="suspend" type="data"
path="${usimpath}/bin/libdlsuspend.so"
datapath="/dat/all/Sample/Equity/SpecialNoticeRR/SpecialNoticeRR_History/JY/${YYYY}/${MM}/${DD}/SuspendNotice.txt" />

Dependency: BaseData

parameter	type	default	description
datapath	string	required	the path of raw data

datatag	type	description
[tag].suspend	Udata_Matrix $<$ int8 $>$	0-not suspend. 1-suspend for 1 hour. 2-suspend for the whole day

Dataloader: FutBaseData

Data loader for base data of futures.

Sample config:

<module id="FutBaseData" type="data"
path="${usimpath}/bin/libdlfutbasedata.so"
datapath="/dat/tmp_all/Futures/dump_${YYYYMMDD}.dat.gz"
settlementclose="false" needtag="false"
realtimedata="" />

Dependency: none

parameter	type	default	description
datapath	string	required	the path of raw data
needtag	bool	true	true if need tag when load data
settlementclose	bool	false	use settlement price as the last close price for intraday futures (caution: this will change last close price significantly )

datatag	type	description
intra_open	Udata_4D $<$ float32 $>$	none
intra_high	Udata_4D $<$ float32 $>$	none
intra_low	Udata_4D $<$ float32 $>$	none
intra_close	Udata_4D $<$ float32 $>$	none
intra_volume	Udata_4D $<$ int32 $>$	none
intra_tvr	Udata_4D $<$ int32 $>$	none
intra_cumvolume	Udata_4D $<$ int32 $>$	cumulative volume from open to current time
intra_cumtvr	Udata_4D $<$ float32 $>$	cumulative turn-over (traded value) from open to current time
intra_openinterest	Udata_4D $<$ float32 $>$	none
intra_askprice1	Udata_4D $<$ float32 $>$	none
intra_askvolume1	Udata_4D $<$ int32 $>$	none
intra_bidprice1	Udata_4D $<$ float32 $>$	none
intra_bidvolume1	Udata_4D $<$ int32 $>$	none
daily_open	Udata_Cube $<$ float32 $>$	none
daily_high	Udata_Cube $<$ float32 $>$	none
daily_low	Udata_Cube $<$ float32 $>$	none
daily_close	Udata_Cube $<$ float32 $>$	none
daily_volume	Udata_Cube $<$ int32 $>$	none
intra_tvr	Udata_Cube $<$ float32 $>$	none
intra_presettle	Udata_Cube $<$ float32 $>$	none
intra_preclose	Udata_Cube $<$ float32 $>$	none
intra_preopeninterest	Udata_Cube $<$ float32 $>$	none
intra_predelta	Udata_Cube $<$ float32 $>$	none
daily_delta	Udata_Cube $<$ float32 $>$	none
daily_openinterest	Udata_Cube $<$ float32 $>$	none
daily_settlement	Udata_Cube $<$ float32 $>$	none
daily_upperlimitprice	Udata_Cube $<$ float32 $>$	none
daily_lowerlimitprice	Udata_Cube $<$ float32 $>$	none
daily_askprice1	Udata_Cube $<$ float32 $>$	none
daily_askvolume1	Udata_Cube $<$ int32 $>$	none
daily_bidprice1	Udata_Cube $<$ float32 $>$	none
daily_bidvolume1	Udata_Cube $<$ int32 $>$	none

Note: when perform daily sim, only load/build daily_* data, and intra_* data will all be empty.

Dataloader: FutIndex

Data loader for index data of futures.

Sample config:

<module id="FutIndex" type="data"
path="${usimpath}/bin/libdlfutindex.so"
datapath="/dat/tmp_all/Futures/dump_${YYYYMMDD}.dat.gz"
adjprices="true" weightbase="daily_openinterest"
minvalue="1000" />

Dependency: FutBaseData

parameter	type	default	description
datapath	string	required	the path of raw data,Usually it is the same value with FutBaseData.
adjprices	bool	false	when true, will calcuate adj prices listed below
weightbase	string	required	the data tag used to calculate weight. must be the tag in FutBaseData of Udata_Cube $<$ float32 $>$ type, e.g. daily_openinterest, daily_tvr
minvalue	float	0	min value of weightbase. The futures with low value will be excluded when calculating index.

datatag	type	description
indexweight	Udata_Cube $<$ float32 $>$	weight of each futures when calc index. indexweight(di, ci, mi).
dailyclose	Udata_Matrix $<$ float32 $>$	close price of index. dailyclose[di][ci]. weighted sum of close price of each mi.
dailyreturn	Udata_Matrix $<$ float32 $>$	weighted sum. this is also used to calculated adj prices.
dailyopen	Udata_Matrix $<$ float32 $>$	weighted sum
dailyhigh	Udata_Matrix $<$ float32 $>$	high of weighted sum of price
dailylow	Udata_Matrix $<$ float32 $>$	low of weighted sum of price
dailyaskprice	Udata_Matrix $<$ float32 $>$	weighted sum of average ask price of each mi.
dailybidprice	Udata_Matrix $<$ float32 $>$	weighted sum of average bid price of each mi.
dailyaskvolume	Udata_Matrix $<$ float32 $>$	sum of average ask volume of each mi. no weight is used.
dailybidvolume	Udata_Matrix $<$ float32 $>$	sum of average bid volume of each mi. no weight is used.
dailyvolume	Udata_Matrix $<$ float32 $>$	sum of volume of each mi. no weight is used.
dailytvr	Udata_Matrix $<$ float32 $>$	sum of tvr of each mi. no weight is used.
dailyopeninterest	Udata_Matrix $<$ float32 $>$	sum of openinterest of each mi. no weight is used.
dailypreclose	Udata_Matrix $<$ float32 $>$	weighted sum
dailypresettle	Udata_Matrix $<$ float32 $>$	weighted sum
dailyupperlimit	Udata_Matrix $<$ float32 $>$	weighted sum
dailylowerlimit	Udata_Matrix $<$ float32 $>$	weighted sum
intraclose	Udata_Cube $<$ float32 $>$	intraday version of dailyclose. intraclose(di, ti, ci)
intraopen	Udata_Cube $<$ float32 $>$	weighted sum
intrahigh	Udata_Cube $<$ float32 $>$	high of weighted sum of price
intralow	Udata_Cube $<$ float32 $>$	low of weighted sum of price
intratvr	Udata_Cube $<$ float32 $>$	sum of tvr in current time slot
intravolume	Udata_Cube $<$ float32 $>$	sum of volume in current time slot
intracumvolume	Udata_Cube $<$ float32 $>$	sum of cummulative (from open to now) volume
intracumtvr	Udata_Cube $<$ float32 $>$	sum of cummulative (from open to now) tvr
intraopeninterest	Udata_Cube $<$ float32 $>$	sum of openinterest
intraaskprice	Udata_Cube $<$ float32 $>$	weighted sum of average ask price
intraaskvolume	Udata_Cube $<$ float32 $>$	sum of average ask volume
intrabidprice	Udata_Cube $<$ float32 $>$	weighted sum of average bid price
intrabidvolume	Udata_Cube $<$ float32 $>$	sum of average bid volume
dailyadjclose	Udata_Matrix $<$ float32 $>$	adjusted index close price.
dailyadjopen	Udata_Matrix $<$ float32 $>$	none
dailyadjhigh	Udata_Matrix $<$ float32 $>$	none
dailyadjlow	Udata_Matrix $<$ float32 $>$	none
dailyadjaskprice	Udata_Matrix $<$ float32 $>$	none
dailyadjbidprice	Udata_Matrix $<$ float32 $>$	none
intraadjclose	Udata_Cube $<$ float32 $>$	none
intraadjopen	Udata_Cube $<$ float32 $>$	none
intraadjhigh	Udata_Cube $<$ float32 $>$	none
intraadjlow	Udata_Cube $<$ float32 $>$	none
intraadjaskprice	Udata_Cube $<$ float32 $>$	none
intraadjbidprice	Udata_Cube $<$ float32 $>$	none
intraadjpreclose	Udata_Cube $<$ float32 $>$	none
intraadjpresettle	Udata_Cube $<$ float32 $>$	none
intraadjupperlimit	Udata_Cube $<$ float32 $>$	none
intraadjlowerlimit	Udata_Cube $<$ float32 $>$	none

Note 1: when adjprices="false", all adj version data is empty and not calculated. That is to say, you can load adj version data with no problems, but will crash when you read it.

For example:

class Usim_AlphaDemo : public Usim_AlphaFutBase
{
public:
    Usim_AlphaDemo(const XmlNode & cfg)    : Usim_AlphaFutBase(cfg),
        adjindexclose(get_data<Udata_Matrix<float32> > ("FutIndex.dailyadjclose"))    //NOTE: this is OK.
    {
        float32 p = adjindexclose[nStartDi][0];    //NOTE: this will crash.
    }

    //your code of alpha

    const Udata_Matrix<float32> & adjindexclose;
}

Note 2: when in daily simulation, all intraday version data not be built or loaded, hence empty. You can load but not read them. The case is similar with Note 1.

Note 3: when raw data is invalid, the corresponding index data is 0, not NAN. This rule applies to all the data in FutIndex.

Note 4: dailyaskprice is a weighted sum, but dailyaskvolume is a simple sum. Thus dailyaskprice * dailyaskvolume might not get a meaningful result. The same case applies to other volume/price pairs.

Dataloader: FutIndexTradeWeight

weight when transfer futures index alpha to normal futures alpha.

Sample config:

<module id="FutIndexTradeWeight" type="data"
path="${usimpath}/bin/libdlfutindextradeweight.so"
indicator="daily_tvr" minvalue="1000" />

Dependency: FutBaseData

parameter	type	default	description
indicator	string	daily_tvr	the datatag of indicating the weight. Data type should be Udata_Cube $<$ float32 $>$
minvalue	float	0	minimum value of indicator

Dataloader: FutTopLiquid

Top liquid for futures.

Sample config:

<module id="AllButIF" type="data"
path="${usimpath}/bin/libdlfuttopliquid.so" allow="all"
disallow="IF" />

Dependency: FutBaseData FutProfile

parameter	type	default	description
allow	string	all	allowed commodities. CASE SENSITIVE. e.g. "al a SM au", "all"
disallow	string	[empty]	disallowed commodities. "all" is not supported.

note: a commodity is in the universe if it is in allow (or "all" in allow) and not in disallow.

Dataloader: FutDominant

Dominant futures.

Sample config:

<module id="FutDominant" type="data"
path="${usimpath}/bin/libdlfutdominant.so" universe="" />

Dependency: FutBaseData

parameter	type	default	description
universe	string	[empty]	futvalid data source. if empty, consider all futvalid is true

datatag	type	description
dominant	Udata_Matrix $<$ int32 $>$	the mi of dominant futures of a ci
nearest	Udata_Matrix $<$ int32 $>$	the mi of nearest futures of a ci

Dataloader: FutProfile

Load multiplier and trading fee rates for futures.

Sample config:

<module id="FutProfile" type="data"
path="${usimpath}/bin/libdlfutprofile.so"
datapath="future_fee.csv" />

Dependency: none

parameter	type	default	description
datapath	string	required	the path of input data

datatag	type	description
multiplier	Udata_Ix $<$ float32 $>$	multiplier of commodities
exchange	Udata_Ix $<$ float32 $>$	exchange number 0-CFFEX 1-SHFE 2-ZCZE 3-DCE
basefee_open	Udata_Ix $<$ float32 $>$	fee that need to pay to open 1 share, in RMB
basefee_close	Udata_Ix $<$ float32 $>$	fee that need to pay to close 1 share, in RMB
basefee_closetoday	Udata_Ix $<$ float32 $>$	fee that need to pay to closetoday 1 share, in RMB
floatfee_open	Udata_Ix $<$ float32 $>$	fee rate that need to pay to open 1 share
floatfee_close	Udata_Ix $<$ float32 $>$	fee rate that need to pay to close 1 share
floatfee_closetoday	Udata_Ix $<$ float32 $>$	fee rate that need to pay to closetoday 1 share

$\mathrm{trading cost} = (\mathrm{basefee} \times \mathrm{trade\_share}) + \mathrm{floatfee} \times \mathrm{trade\_value}$ usually close and closetoday are the same.