Wednesday, 7 November 2012

Candlesticks - burning at both ends

Candlesticks have proven to be pretty complex and really quite interesting. (I've also been on holiday).

The whole premise of Candlesticks is that the relationship between the open, high, low and close of a stock in the previous 2 to 3 periods (days in our case), indicates, through the use of pattern recognition, probabilities of rises.

So my thinking is this:

1. calculate the Markov Chains for each period indicating the:
-1a. 3 open/close situations (3 periods, today, yesterday and the day before and if the open is above or below the close),percentage increase over the subsequent 5 days
-1b. 3 individual indicators stating the usual Candlestick derivatives 
-1c. 2 relative indicators detailing the relationship between the periods
-1d. the percentage increases for the subsequent 5 days

(1b, 1c and 1d taken together with an indicated rise or fall can be thought of as the well known Candlestick indicators that have been observed to indicate  patterns that if popular have been given names (hanging-man lines, dark-cloud covers, evening stars, etc))

2. Calculate the daily Candlestick indicators and compare to the Markov Chains created previously

3. Tweet results

The Markov Chains are currently set at greater than 11.5% rise for greater than 80% of the time for the subsequent 5 days, and yield results such as:

72.4         uuu0202020804       BAC           6       100       6       0
81.42       uuu0202021808       BAC           5       100       5       0
88.83       uuu0202021313       NLMA       181       98.34       178       3
33.79       uuu0202021313       IHTD         38       97.37       37       1
73.63       uuu0202020808       BAC          34       97.06       33       1
49.4         uuu0303031212       EEE           32       96.88       31       1
121.01     uuu0202021313       YKBD       291       96.56       281       10

The columns indicate, the percentage increase, the Candlestick indicator, the company, the number of times this indicator has triggered, the percentage that this indicator has been over 11.5%, the number of times it has been over the 11.5% and the number of times it has been less than 11.5%.

So we see above, finding an indicator for the company NLMA of uuu0202021313 today would indicate a 98.34% chance that over the next 5 days this stock will rise more than 11.5%, but the current average percentage increase is 88.83% for a result set of 181 times this Candlestick indicator has been triggered.

The '...' indicate that this is in the middle of a report.
Lets see some interesting Candlesticks that indicate a rise 100% of the time:

22.56       uuu0202021818       LTHP       22       100       22       0
34.29       uuu0505051717       CRL       14       100       14       0
15.84       uuu0202021313       GLDE       12       100       12       0
34.34       uuu0202021919       INDG       9       100       9       0
16.5         ddd0505051818       HAMP       9       100       9       0
37            uuu0202021819       INDG       8       100       8       0
159.43     uuu0202022004       BAC       8       100       8       0
36.97       uuu0202021918       INDG       7       100       7       0
15.99       uuu0202021413       GLDE       7       100       7       0
73.96       uuu0602020108       BAC       7       100       7       0
16.97       ddd0404041313       PCH       7       100       7       0
32.91       uuu0202021414       CMCI       7       100       7       0

Even though we get figures on LTHP, I cannot find this company on LSE, but looking at CRL (Creightons), for uuu0505051717, this should indicate a greater than 11.5% increase over the subsequent 5 days, and currently an average increase of 34.29% over the last 14 times this has occurred.

From the above, one can see that the first 3 period indicators are usually either all up or all down  - as an example, for PCH (Pochins Share Price (PCH)), just for these 3 indicators, if the close is lower than the open on 3 successive days, and that the other indicators are 04,04,04,13,13; then we currently have a 100% chance (out of a set of 7 instances) that PCH will rise more than 11.5% over the next 5 days, with the current average being a price rise of 16.97%.

I am now tweeting these under the indicator 'LSE_cs' - see at the top for the Twitter page.

Friday, 2 November 2012

Oracle ACL

New security rules on Oracle 11g predicate that one must setup an ACL (Access Control List) to access outside websites. To to this, follow the following setup scripts:

acl => 'utl_http.xml',
description => 'http access',
principal => 'YOUR_SCHEMA',
is_grant => TRUE,
privilege => 'connect');
acl => 'utl_http.xml', 
principal => 'YOUR_SCHEMA',
is_grant => TRUE, 
privilege => 'connect',
position => null);
acl => 'utl_http_mlee.xml', 
host => 'your_hostname');

Sunday, 30 September 2012

Tweeting directly from Oracle

Finally set up the bespoke Oracle Twitter using OAuth rather than using RSS/XML from Twitterfeed.

{easy once you've spent a couple of days working out what the hell is all the fuss about OAuth}

The steps are:

1. set up an application on your Twitter account here

2. make a note of the consumer and account keys and secrets

3. in Oracle create urlencode function:

create or replace function urlencode( p_str in varchar2 ) return varchar2
    /* FROM:
     * KUDOS!
    l_tmp   varchar2(6000);
    l_bad   varchar2(100) default ' >%}\~];?@&<#{|^[`/:=$+''"';
    l_char  char(1);
    for i in 1 .. nvl(length(p_str),0) loop
        l_char :=  substr(p_str,i,1);
        if ( instr( l_bad, l_char ) > 0 )
            l_tmp := l_tmp || '%' ||
                            to_char( ascii(l_char), 'fmXX' );
            l_tmp := l_tmp || l_char;
        end if;
    end loop;

    return l_tmp;

4. update the XXXXX codes below with the relevant values from step 1 and run code within an SQL worksheet window - create a package from this, go on you know how - :




  l_oauth_request_token_url CONSTANT VARCHAR2 (500) := '';
  l_oauth_consumer_key CONSTANT VARCHAR2 (500) := 'XXXX';
  l_oauth_token  CONSTANT VARCHAR2 (500) := 'XXXXX';
  l_oauth_secret CONSTANT VARCHAR2 (500) := 'XXXXX';
  l_oauth_nonce VARCHAR2 (500);
  l_oauth_signature_method CONSTANT VARCHAR2 (10) := urlencode ('HMAC-SHA1');
  l_oauth_timestamp VARCHAR2 (100);
  l_oauth_version CONSTANT VARCHAR2 (5) := urlencode ('1.0');
  l_oauth_consumer_secret CONSTANT VARCHAR2 (500) := 'XXXXXX';

  l_http_method VARCHAR2 (5) := 'POST';
  l_oauth_base_string VARCHAR2 (2000);

  l_oauth_key VARCHAR2 (500) := l_oauth_consumer_secret || '&' || l_oauth_secret ;

  l_sig_mac RAW (2000);
  l_base64_sig_mac VARCHAR2 (100);

  http_req UTL_HTTP.req;
  http_resp UTL_HTTP.resp;

  l_update_send VARCHAR2(2000);
  l_oauth_header  VARCHAR2(2000);

  l_line  VARCHAR2(1024);

  resp_name  VARCHAR2(256);
  resp_value VARCHAR2(1024);

  l_content varchar2(140) := urlencode('hello world');

  l_random varchar2(15);


select dbil.urlencode((sysdate - to_date('01-01-1970','DD-MM-YYYY')) * (86400)) into
l_oauth_timestamp from dual;

  -- RANDOM oauth_nonce
  SELECT dbms_random.string('A', 15)
  INTO l_random

  SELECT dbil.urlencode (UTL_ENCODE.base64_encode(UTL_I18N.string_to_raw (l_random, 'AL32UTF8')))
  INTO l_oauth_nonce

  l_oauth_base_string := l_http_method
                          || '&'
                          || urlencode (l_oauth_request_token_url)
                          || '&'
                          || urlencode ( 'oauth_consumer_key'
                              || '='
                              || l_oauth_consumer_key
                              || '&'
                              || 'oauth_nonce'
                              || '='
                              || l_oauth_nonce
                              || '&'
                              || 'oauth_signature_method'
                              || '='
                              || l_oauth_signature_method
                              || '&'
                              || 'oauth_timestamp'
                              || '='
                              || l_oauth_timestamp
                              || '&'
                              || 'oauth_token'
                              || '='
                              || l_oauth_token
                              || '&'
                              || 'oauth_version'
                              || '='
                              || l_oauth_version
                              || '&'
                              || 'status'
                              || '='
                              || l_content);
  DBMS_OUTPUT.put_line (l_oauth_base_string);

  l_sig_mac := dbms_crypto.mac (  utl_i18n.string_to_raw (l_oauth_base_string, 'AL32UTF8')
                                , DBMS_CRYPTO.hmac_sh1
                                , UTL_I18N.string_to_raw (l_oauth_key, 'AL32UTF8'));
  DBMS_OUTPUT.put_line ('Combined sig: ' || l_oauth_key);
  l_base64_sig_mac := UTL_RAW.cast_to_varchar2 (UTL_ENCODE.base64_encode (l_sig_mac));

  DBMS_OUTPUT.put_line ('MAC Signature (Base64-encoded): ' ||  l_base64_sig_mac);

  l_update_send := l_oauth_request_token_url || '?status=' || l_content;
    http_req := UTL_HTTP.begin_request (  l_update_send
                                        , l_http_method
                                        , UTL_HTTP.http_version_1_1);
   DBMS_OUTPUT.put_line ('UPDATE URL ' || l_update_send);
   UTL_HTTP.set_response_error_check (TRUE);
   UTL_HTTP.set_detailed_excp_support (TRUE);

    l_oauth_header := 'OAuth oauth_nonce="' || l_oauth_nonce || '", '
                      || 'oauth_signature_method="'|| l_oauth_signature_method || '", '
                      || 'oauth_timestamp="'|| l_oauth_timestamp || '", '
                      || 'oauth_consumer_key="'|| l_oauth_consumer_key || '", '
                      || 'oauth_token="' || l_oauth_token || '", '
                      || 'oauth_signature="' || urlencode (l_base64_sig_mac) || '", '
                      || 'oauth_version="' || l_oauth_version || '"';
    utl_http.set_header ( r => http_req,
                          NAME => 'Authorization', VALUE => l_oauth_header);
    DBMS_OUTPUT.put_line  ('HEADER: ' || l_oauth_header);                      
    utl_http.write_text(  r => http_req, DATA => l_content);
    http_resp := utl_http.get_response(r => http_req);
   FOR i IN 1..utl_http.get_header_count(http_resp) LOOP
    utl_http.get_header(http_resp, i, resp_name, resp_value);
    dbms_output.put_line(resp_name || ': ' || resp_value);
  DBMS_OUTPUT.put_line('Getting content:');
        utl_http.read_line(http_resp, resp_value, TRUE);
      END LOOP;
      WHEN utl_http.end_of_body THEN
        DBMS_OUTPUT.put_line('No more content.');

   utl_http.end_response(r => http_resp);

    when others then
      DBMS_OUTPUT.put_line('HTTP ERROR: ' || utl_http.get_detailed_sqlerrm);



And that's it - you should be tweeting away from your Oracle Database - comment if you have any difficulties.

I am in the process now of changing the twitter feeds to contain all information rather than just the top 5.

Thursday, 27 September 2012


Just back from hols and will be documenting here the new Candlestick/Markov/Trace analysis that I have been working on - isnt that what holidays are for ?

But as a quick indicator, I am up to my eighth straight 11.5%; just had TOM, WSG, SEA and BRT.

Wednesday, 5 September 2012

AHT and new system

AHT did rather well yesterday, I was out at 11.5%, but its still going up.

New system delivered and setup - to the right is the new system screen shot of processor cores under calc mode - oasis is parallel processing

Currently into SEA, KMG, SIE, SLE.

Saturday, 1 September 2012


Today's AHT success coefficients are 1900/221 which are pretty good

Friday, 31 August 2012

even more trace analysis

Well this is interesting. All of the reports below (apart from HCM, this is included as a demo) are current live indicators for today - 31st-Aug-2012.
The columns in order left to right are:
company, date tweeted, opening price for the next day, 5, 10 and 15 day percentage increases, tweedledee, tweedledum (two success coefficients)
(empty columns denote that we do not yet know the opening price because today's data has not been analysed - it not being the end of day today yet)

I have noticed that any tweedledee below 200, with a low tweedledum (less than about 100) more often than not indicates a mediocre performance in our given time frame:

FTF 30-AUG-12

189 0
FTF 29-AUG-12 86.25 0 0 0 189 0
FTF 28-AUG-12 85 1.47 1.47 1.47 189 0
FTF 24-AUG-12 87 0 0 0 153 0
FTO 30-AUG-12

27 1000
FTO 29-AUG-12 9.4 1.65 1.65 1.65 27 1000
FTO 28-AUG-12 9.5 0.58 0.58 0.58 27 1000
FTO 24-AUG-12 9.3 3.23 3.23 3.23 27 1000
FTO 23-AUG-12 9.5 1.05 1.05 1.05 27 1000

On the other hand a large tweedledee denotes a good performance indicator within our time frame:

HCM 03-AUG-12 410 6.1 6.1 6.1 2481 705
HCM 02-AUG-12 400 8.75 8.75 8.75 2481 705
HCM 01-AUG-12 405 7.41 7.41 7.41 2480 704
HCM 31-JUL-12 390 11.54 11.54 11.54 2480 704
HCM 30-JUL-12 366 18.85 18.85 18.85 2377 692
HCM 27-JUL-12 345 23.19 26.09 26.09 2377 692
HCM 26-JUL-12 346 21.39 25.72 25.72 2377 692
HCM 25-JUL-12 350 17.14 24.29 24.29 2377 692
HCM 24-JUL-12 355 14.08 22.54 22.54 2377 692
HCM 23-JUL-12 366 10.66 18.85 18.85 2377 692

But as you can see it is also dependent on time - the success coefficients move in a wave like form, and there is an optimal time when we should trade.

(Also tweedledee has to be taken with tweedledum)

The two above statements together make for an interesting concept - as demonstrated below:



What we see here is that SIE has been trundling away not doing anything in particular but all of a sudden the success coefficients have increased. I am guessing that SIE should have been bought on the opening of the 29th Aug 2012.

As a bonus for today AHT has just triggered today with a tweedledee of 1865 and a tweedledum of 208.

Thursday, 30 August 2012

traces - more results

So here we are, where each night we compare the 300,000 odd Markov chains to the indicators triggered for each individual company.

To the right is a screen shot of all the LSE tweets around a randomly chosen date around 1st Aug this year - counting backwards the columns are: last column, success coefficient tweedledum, is the average number of time each individual indicator has occurred above the required 2 (the numbers are in 0.000s and so 231 would mean 2.0231 ), the penultimate column is the number of times separate individual indicators have been triggered, success coefficient tweedledee, here 561 means 561. The next three columns are percentage increase over the last 5, 10 and 15 days (reading left to right), The first column is the company, then the date and then the opening price of the next day (approximate price at which we would buy).

Currently analyzing these results, but have been trading high tweedledee and tweedledum numbers and it has been working (EXI, AGTA, HUM, SXX (although I missed buying this one, but it did act as predicted))

Wednesday, 22 August 2012


Fibonacci is another buy/sell indicator sort of thing and am looking at incorporating - it is one where the first statement is 'if up trend do this, and if down trend do the other' (the same as Candlestick calcs) as well as pin pointing tops and bottoms of trends - luckily I have been working a long time trying to implement tops and bottoms, and am piggy backing our trace system onto Fibonacci -... as well as working on the 5.3b Candlesticks calculations (first step below, new bigger faster system)

First up, calculate the Markov chains for Fibonacci retracements.
Second, do something to get 5.3b Candlestick calculations to a reasonable time.

Sold AGTA into TOM just now on LSE.

new system

Awaiting delivery of new system -  the interesting bits being:
Processor (CPU)Intel® Xeon® 8-Core E5-2687W (3.1 GHz, 8.00 GT/s, 20M L3 Cache)
MotherboardASUS® P9X79 WS - SOCKET 2011, QUAD DDR3, USB 3.0, SATA 6 GB/s
Memory - 1st Hard Disk240GB INTEL® 520 SERIES SSD, SATA 6 Gb/s (upto 550MB/sR | 520MB/sW)
2nd Hard Disk240GB INTEL® 520 SERIES SSD, SATA 6 Gb/s (upto 550MB/sR | 520MB/sW)
3rd Hard Disk240GB INTEL® 520 SERIES SSD, SATA 6 Gb/s (upto 550MB/sR | 520MB/sW)
4th Hard Disk240GB INTEL® 520 SERIES SSD, SATA 6 Gb/s (upto 550MB/sR | 520MB/sW)

Multi disks and processor cores for the concurrent processing of Candlesticks

Tuesday, 14 August 2012

lets do 5.3 billion Candlestick calculations

Been heavily involved in trying to condense the Candlestick processing. The story so far is that I have 8 Candlestick indicators, 3 of which have 2 values and the remaining 5 have 22 values. What this gives is 256 divided by 2 combinations (each candlestick option appears only in half of the options) of 22 to the power 5 times 2 to the power 3, giving 5,277,319,168 separate Candlestick calculations - as the previous indicator calculations take 2 hours per company for 300,000 odd calculations, then 5.3b will take approximately 4 years per company to complete, there are approximately 2,500 companies in the LSE, and so we should be finished by about the year 12012. This is unacceptable, and so thinking up ways to optimise.  

Monday, 6 August 2012


Came out of EXI with the required 11.5%, but then got taught a valuable lesson in trading with SXX - SXX via L2 was trading nicely with 14.5 as buy and 14.7 as sell, so I thought I'd go in at 14.6, as one would. But in a rising trend there were no takers for 14.6 and so lost out on todays high of 16.0 (at end of day) - I guess the lesson is that on a rising trend go with the sellers numbers.

SXX is not triggered this evening, but WSG and VTBR are - as are some others that are on the tweet. WSG tomorrow.

Thursday, 2 August 2012

Friday, 20 July 2012

tweedle dee or tweedle dum

An interesting fight is currently being waged between 2 of my success coefficients - with the current proponents being QPP and SNRP (on the LSE).

Both companies had been tweeted yesterday.

The first coefficient relates to the number of trace formulae triggered; the second relates to the number of times the formulae have triggered in the past averaged over the triggered formulae.

The numbers respectively are:

QPP      37        2.946
SNRP    233     2.09

Of note are that all of the numbers for each company are outstanding, 233 is a very high number (and if you remember we carry out around 300,000 per company), with 2.946 also being a respectful number (and has the meaning that from all of the 37 formulae that were triggered yesterday, the number of times these were triggered and reached an 11.5% increase over the subsequent 5 days was 2.946, to get more than 2 triggers is extraordinary)

But lets say we need to go for one, which one to choose. Better number of triggers or better times triggered, or a relationship between these.

Currently today QPP is up 5.77% and SNRP is up 2.33%, so at the moment it is coming down on the side of tweedle dum (or success coefficient number 2), but I am now looking at the relationship between these numbers.

To complete the set for yesterday and as a note, the remaining tweets were:

PVCS  145  2.000
STCM  31   2.000
AGY     4    2.000
CYAN 15   2.000

For reference I have added these two success coefficients onto the tweeted information

Tuesday, 17 July 2012

all are equal

... but some are more equal than others.

The trace tweets are now ordered by not only the trace algorithm but also the success coefficients - if one had all the triggered results this would not matter, but the auto process only tweets five tweets - so now with added goodness at the top.

Top of the list for LSE is something called RBPI but as this doesn't seem to exist on either LSE shares nor Google Finance, the top one today for LSE is AST.


Delving deeper into Markov, and I see that the answer to the question to 'who uses Markov?' below is that oasis already uses Markov Chains to determine Stocks to pick.

Markov looks at individual states and presents percentage probabilities of outcomes. If we think of oasis indicators as states then the reporting system is producing the Markov products from the probability Chains.

Markov nicely formalized the generalized situations and gives a way to visualize what is going on.

The Markov Chain that oasis is trying to formalize is pretty specific and quite complex, by looking at the state of indicators in the current period and comparing to a desired outcome with maximal probability.

At least what I am trying to do has a name

Monday, 16 July 2012


And there was I thinking that what I was doing was akin to balancing an elephant on the edge of a razor blade. Markov is suggesting balancing said elephant on the tip of a needle; a lot of analysis and formulae have been composed in order to get predictions from the least amount of data; do people really use Markov Switching Multifractal ?, cos it is essentially, at least from my initial analysis, saying something to the effect 'what ever the price is now, take a prediction within a gaussian curve as to the price tomorrow, with the gaussian curve being derived by past analysis of the price movements, and dependent on volatility' - I am still going to program it in and see what comes out... Markov on the stochastics on the psar on the vi on close... sounds like something fun I need to do.

Anyone have any views on Markov ?

Markov Chains and Switching Multifractals

On my last research outing I came across Markov chains and switching multifractals and am looking into programming these in to supplement the trace and candlesticks.

its not 5 days

... but its pretty close.

As we can see from the below table, we've done pretty well on the 29th Jun to now, and the companies highlighted since then are looking encouraging.

Columns are LSE company, date of buy trigger, next days opening price, % increase over 5 days and % increase in 10 days

PDX   10-JUL-12   10.25      9.15    9.15
AGY   10-JUL-12   7.5      4.00    4.00
AUP3   10-JUL-12   6698      0.00    0.00
AST   10-JUL-12   2.405      0.83    0.83
PDX   09-JUL-12   10      11.88    11.88
AGY   09-JUL-12   7.55      3.31    3.31
AUP3   09-JUL-12   6744      0.14    0.14
RXP   09-JUL-12   3.5      5.71    5.71
SIE   06-JUL-12   66.4434      3.53    3.53
T47   06-JUL-12   122.838      9,898.13    9,898.13
PDX   06-JUL-12   10.5      6.55    6.55
AUP3   06-JUL-12   6658.95      1.42    1.42
RXP   06-JUL-12   3.525      4.96    4.96
SIE   06-JUL-12   66.4434      3.53    3.53
SIG   06-JUL-12   2856      0.49    0.49
PDX   06-JUL-12   10.5      6.55    6.55
CKSN   06-JUL-12   600      0.58    0.58
RXP   06-JUL-12   3.525      4.96    4.96
SIE   06-JUL-12   66.4434      3.53    3.53
SIG   06-JUL-12   2856      0.49    0.49
PDX   06-JUL-12   10.5      6.55    6.55
AGQ   06-JUL-12   14.32      8.24    8.24
CKSN   06-JUL-12   600      0.58    0.58
DNO   06-JUL-12   561.5      1.25    1.25
RXP   06-JUL-12   3.525      4.96    4.96
WSG   06-JUL-12   30.5      1.64    1.64
SIE   04-JUL-12   67.2      2.37    2.37
SIG   04-JUL-12   2770      7.65    7.65
PDX   04-JUL-12   10      11.88    11.88
AGQ   04-JUL-12   16.14      2.23    2.23
CKSN   04-JUL-12   613.5      1.87    1.87
DNO   04-JUL-12   557      2.06    2.06
RXP   04-JUL-12   3.7      1.35    1.35
WSG   04-JUL-12   31.5      4.29    4.29
SIE   03-JUL-12   67.36      2.02    2.12
SIG   03-JUL-12   2923      2.02    2.02
PDX   03-JUL-12   11.75      0.00    0.00
AGQ   03-JUL-12   16.8      0.65    0.65
CKSN   03-JUL-12   611      2.29    2.29
FUM   03-JUL-12   77.5      0.65    0.65
DNO   03-JUL-12   564      0.80    0.80
RXP   03-JUL-12   3.6944      1.50    1.50
WSG   03-JUL-12   33      0.00    0.00
SIE   02-JUL-12   67.145      2.35    2.45
SIG   02-JUL-12   2876      3.69    3.69
PDX   02-JUL-12   11.75      3.19    3.19
AGQ   02-JUL-12   15      13.17    13.17
IBTM   02-JUL-12   128.42      1.93    2.69
CKSN   02-JUL-12   605      3.31    3.31
FUM   02-JUL-12   77      1.30    1.30
WSG   02-JUL-12   34.35      0.44    0.44
SIE   29-JUN-12   66.65      3.11    3.21
PDX   29-JUN-12   12.5      4.00    4.00
AGQ   29-JUN-12   14.25      19.12    19.12
IBTM   29-JUN-12   128.2      2.11    2.87
CKSN   29-JUN-12   596      4.87    4.87
FUM   29-JUN-12   74.148      5.20    5.20
WSG   29-JUN-12   33.55      2.83    2.83
SIE   29-JUN-12   66.65      3.11    3.21
AGOL   29-JUN-12   5.5      0.00    0.00
PDX   29-JUN-12   12.5      4.00    4.00
AGQ   29-JUN-12   14.25      19.12    19.12
IBTM   29-JUN-12   128.2      2.11    2.87
CKSN   29-JUN-12   596      4.87    4.87
FUM   29-JUN-12   74.148      5.20    5.20
WSG   29-JUN-12   33.55      2.83    2.83
LOGP   29-JUN-12   34.45      48.04    74.17
YOU   29-JUN-12   67.5      0.00    0.00
SIE   29-JUN-12   66.65      3.11    3.21
PRX   29-JUN-12   365      2.60    2.60
PVR   29-JUN-12   509      23.38    26.72
VUKE   29-JUN-12   26.07      10,068.78    10,068.78
RBS   29-JUN-12   216.6      1.99    1.99
IQE   29-JUN-12   27.5      15.09    15.09
BEM   29-JUN-12   11.775      1.91    1.91
ATC   29-JUN-12   .2838      16.28    16.28
FRR   29-JUN-12   .53      3.77    7.55
ANGM   29-JUN-12   1.04      0.00    0.00
GCM   29-JUN-12   42      64.74    64.74
BMR   29-JUN-12   3.195      23.63    23.63
FQM   29-JUN-12   1093.063      15.41    15.41
CAD   29-JUN-12   30.5      0.54    0.54
AGOL   29-JUN-12   5.5      0.00    0.00
ARG   29-JUN-12   15.4475      10.51    25.59
DEO   29-JUN-12   23      3.26    6.52
CRW   29-JUN-12   255      17.55    17.55
LDP   29-JUN-12   .24      70.00    70.00
AFC   29-JUN-12   21.43      16.57    16.57
DDV1   29-JUN-12   55.1      8.89    8.89
SER   29-JUN-12   1.602      8.93    11.42
SOU   29-JUN-12   1.01      17.82    17.82
IBGL   29-JUN-12   130.9      0.70    0.70

We can discount VUKE and T47.

LDP, LOGP, ARG, PVR and PDX are notable

Wednesday, 4 July 2012

trace results up to now

Following is a table of the companies that have been tweeted over the last 2 working days (and since the date I have been taking a history of what is being tweeted, rather than just looking at the tweets), the columns are date of tweet, company name, next days open price, and the last column is the percentage increase over the subsequent 5 days (ie over the last 2 days (not counting today because we arent finished yet))

We would have done very well with LDP if we had traded on the next day after the 29th Jun !

I will post again when I have more results

There are duplicates per day because oasis tweets twice a day

And from the numbers, this is why I think the traces look like they are working

29 June was a day of testing the success coefficients for LSE and hence the large number of company tweets - have left them in because I am interested in the outcome in 3 days time

02-JUL-12 SIE   67.145     1.08
02-JUL-12 SIG   2876     1.77
02-JUL-12 PDX   11.75     3.19
02-JUL-12 AGQ   15     13.17
02-JUL-12 IBTM   128.42     .16
02-JUL-12 CKSN   605     1.82
02-JUL-12 FUM   77     1.3
02-JUL-12 WSG   34.35     .44
29-JUN-12 SIE   66.65     2.03
29-JUN-12 PDX   12.5     4
29-JUN-12 AGQ   14.25     19.12
29-JUN-12 IBTM   128.2     .51
29-JUN-12 CKSN   596     3.36
29-JUN-12 FUM   74.148     5.2
29-JUN-12 WSG   33.55     2.83
29-JUN-12 SIE   66.65     2.03
29-JUN-12 AGOL   5.5     0
29-JUN-12 PDX   12.5     4
29-JUN-12 AGQ   14.25     19.12
29-JUN-12 IBTM   128.2     .51
29-JUN-12 CKSN   596     3.36
29-JUN-12 FUM   74.148     5.2
29-JUN-12 WSG   33.55     2.83
29-JUN-12 LOGP   34.45     13.21
29-JUN-12 YOU   67.5     0
29-JUN-12 SIE   66.65     2.03
29-JUN-12 PRX   365     0
29-JUN-12 PVR   509     5.26
29-JUN-12 VUKE   26.07     1.48
29-JUN-12 RBS   216.6     1.99
29-JUN-12 FRR   .53     3.77
29-JUN-12 IQE   27.5     11
29-JUN-12 BEM   11.775     1.91
29-JUN-12 ATC   .2838     16.28
29-JUN-12 GCM   42     2.38
29-JUN-12 ANGM   1.04     0
29-JUN-12 FQM   1093.063     11.16
29-JUN-12 BMR   3.195     20.47
29-JUN-12 CAD   30.5     .54
29-JUN-12 AGOL   5.5     0
29-JUN-12 ARG   15.4475     .34
29-JUN-12 DEO   23     3.26
29-JUN-12 CRW   255     10.2
29-JUN-12 LDP   .24     58.33
29-JUN-12 AFC   21.43     2.17
29-JUN-12 SER   1.602     3
29-JUN-12 IBGL   130.9     0
29-JUN-12 SOU   1.01     8.91

Tuesday, 3 July 2012


03-JUL-12 LON LSE_trace :0 WSG
03-JUL-12 LON LSE_trace :0 RXP
03-JUL-12 LON LSE_trace :0 DNO
03-JUL-12 LON LSE_trace :0 FUM
03-JUL-12 LON LSE_trace :0 CKSN
03-JUL-12 LON LSE_trace :0 AGQ
03-JUL-12 LON LSE_trace :0 PDX
03-JUL-12 LON LSE_trace :0 SIG
03-JUL-12 LON LSE_trace :0 SIE

All on LSE, and all today ready for tomorrow

Monday, 2 July 2012


02-JUL-12 LON LSE_trace :0 WSG
02-JUL-12 LON LSE_trace :0 FUM
02-JUL-12 LON LSE_trace :0 CKSN
02-JUL-12 LON LSE_trace :0 IBTM
02-JUL-12 LON LSE_trace :0 AGQ
02-JUL-12 LON LSE_trace :0 PDX
02-JUL-12 LON LSE_trace :0 SIG

All on the LSE and all via the trace, for today ready for tomorrow

FUM sounds like a fun company

Friday, 29 June 2012

exchange traces

Through making the NYSE, NASDAQ and LSE traces work, I have spotted significant differences in the way the separate exchanges seem to operate. The same trace formulae are being run against each exchange, the combinations are full and complete and so there are no other options here, but the different way each exchange reacts to these formulae is startling.

The tweets for the traces are the resultant per day triggers of formulae with the best success coefficients. It is the success coefficients that are different for each of the exchanges which needed to be updated for NYSE and NASDAQ because the LSE one, used in the first instance,  produced zero results.

Wednesday, 27 June 2012

Tweet recap and update

The content of the tweets has changed a little over the last 3 months and needs a little explanation.

Companies that have been triggered by a formula are tweeted twice a day, just after market close and analysis, and 12 hours later, to ensure that the most current companies are tweeted at the most current time.
The Auto tweets API allows only 5 companies to be tweeted. If you would like to see the full list, email.

The initial beta formulae, crafted by hand over weeks and months, are still being tweeted. These are designated with titles such as LSE_b4. These are good, and worth consideration.

The trace tweets have titles such as LSE_trace and if you do nothing else, consider these. These are the companies that have had specific bespoke algorithms run against them (around 300,000 in total) to obtain the best matches and if these trigger then we have a good chance.
oasis is about two thirds into the completion of the full trace analysis (8 weeks processing in total, 24 hours per day, 5 weeks in), but as each company completes analysis the trace formulae are put onto the trigger list for that company.
I am currently testing the traces for the NYSE and NASDAQ at the moment.

Tweets that say 'no companies found for analysis'/'no co' mean that the trigger did not return any companies following the specific formula for that day, and just contains a link back to here.

The tweets include a link to Google Finance for the company being tweeted.

And to recap on the analysis techniques, these currently are:
1. trace - combination analysis and bespoke company formula - explanation here
2. beta - hand crafted formula - explanation here
3. Candlesticks - explanation here

And also, currently on my list of things to do, in no particular order, are:
1. Add FX into exchanges
2. Complete Candlestick analysis
3. Link into RNS for LSE
4. NYSE and NASDAQ traces


Here's a coincidence. SIE has just come up on the trace tweets for today.

(Along with WSG, PDX, AGQ and AGOL - AGOL we have seen before)

Sunday, 24 June 2012

initial Candlestick analysis

Below is a section of the report of my initial analysis of the Candlestick processing. Currently the report only includes elemental and relational aspect of Candlesticks - the inclusion of the trend, or in fact any aspect to the usual indicators such as rsi, adx, vi or stochastics, etc is being analysed at the moment.

The columns are - LSE company (my tests are being carried out on the London exchange LSE, I will be adding NYSE and NASDAQ very shortly), code combinations of elements and relationals, number of time this has occurred in the last 5 years with a subsequent 5 day rise of over 11.5%, and the number of times in total this combination has occurred for this company.

The report is sorted by the relationship between number of times triggered and producing an 11.5% increase, and the total number of times triggered


T27  uuuE004F004G004R018S018  5 5
SIE  uuuE002F002G002R008S006  4 4
SIE    uuuE002F002G002R008S008  4 4
SAUS uduE002F002G002R014S008  4 4
SER   uuuE004F004G001R017S017  4 4
AAZ    uuuE004F004G004R012S012  3 3
H50E   duuE002F002G002R008S001  3 3
RUKS uddE003F002G002R014S002  3 3
BOR uduE003F003G003R013S013  3 3
TDK uuuE002F002G002R014S014  3 3
OPG uuuE003F002G003R013S001  3 3
T17   uuuE004F004G004R018S019  3 3
SIE    uuuE002F002G002R013S014  3 3
T17   uuuE004F004G004R019S018  3 3
XEL  uuuE002F002G002R013S013  3 3
SAUS  uddE002F002G002R013S009  3 3
BEM uuuE002F004G004R012S012  3 3
FRR  dddE002F002G002R014S009 3 3
EME uudE003F002G003R013S013  2 2
SAUS  uuuE004F003G003R001S002  2 2
SER  uuuE004F001G004R017S017  2 2
AFC  dddE002F002G007R002S021 2 2
COP uudE004F004G004R017S019  2 2
EME uuuE003F004G004R012S003  2 2
CAZA  uuuE003F003G002R001S002  2 2
RICI  uudE003F003G006R001S014  2 2


So lets take the first sensible line - SIE and 'uuuE002F002G002R008S006'

(T27, even though we get daily data downloaded from the LSE, T27 is neither listed in LSE Shares or on Google Finance, so do not really know what this is)

I believe the Candlestick code combinations like 'uuuE002F002G002R008S006', or part thereof, have been noticed and have been given names - if we delve a little deeper, we notice patterns emerging from the SIE record, for example for SIE, all the combinations for the above report section all start with 'uuuE002F002G002' and the pattern changes only after the R code - you can probably guess what each 3 letter code means, but what will be less obvious is how each 3 letter code relates to the Candlestick formulae.

What I can say at this point is that the codes for each company are now being analysed Candlestick wise and compared with the current day codes, any that follow the rule greater than 90% of the time will be tweeted.

Also, there are two other aspects to the work on Candlesticks that have spawned separate analysis
1. Not only has this analysis given the 11.5% risers, but also the codes to follow for each company with respect to decliners - ie analysis on sells
2. A serious and complex link into the traces that are happening at the moment - Candlesticks can be thought of as another indicator, rather than a separate entity

I think I'll call 'uuuE002F002G002'... 'two thumbs'

Wednesday, 20 June 2012

trace update

Quick update on the traces being run at the moment - LSE:762:41.5m, NYSE:471:7.7m and NASDAQ:655:4.6m
(exchange:number of companies traced:number of records)

Revised estimate to completion is now 8 weeks, we have had 4.5

the 22 Candlestick relational formulae

And for interest, following are the 22 definitive relational options for Candlesticks

xxx is the current period, yyy is the previous period, aaa is open price and bbb is the close price (specifically for the relational formulae for 2 periods both of which have the close greater than the open, aaa and bbb will be different for the other combinations of close and open price)

Each statement is taken separately and is period mutually exclusive, so as an example the first statement 'and xxx.f_low>= yyy.f_high' means 'if todays low is greater than or equal to yesterdays high' - with this statement one does not need to go any further because if this is true then we already know all the other relational relationships, ie that today's close, open and high will also be greater than or equal to yesterday's high

Below is also a screen shot of the logic by which one can get to the formulae, and should help to make some sense of the formulae


and xxx.f_low >= yyy.f_high

and xxx.f_low < yyy.f_high and xxx.f_high < yyy.f_low

and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low < yyy.f_low and >= and >= yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low < yyy.f_low and >= and < yyy.bbb and xxx.bbb >= yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low < yyy.f_low and >= and < yyy.bbb and xxx.bbb < yyy.bbb

and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low < yyy.f_low and < and >= yyy.f_low and xxx.bbb >= yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low < yyy.f_low and < and >= yyy.f_low and xxx.bbb < yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low < yyy.f_low and < and < yyy.f_low and xxx.bbb >= yyy.f_low and xxx.bbb >= and xxx.bbb >= yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low < yyy.f_low and < and < yyy.f_low and xxx.bbb >= yyy.f_low and xxx.bbb >= and xxx.bbb < yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low < yyy.f_low and < and < yyy.f_low and xxx.bbb >= yyy.f_low and xxx.bbb <
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low < yyy.f_low and < and < yyy.f_low and xxx.bbb < yyy.f_low

and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= and >= and xxx.bbb >= yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= and >= and xxx.bbb < yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= and < and xxx.bbb >= yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= and < and xxx.bbb < yyy.bbb

and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= yyy.f_low and >= and >= yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= yyy.f_low and >= and < yyy.bbb and xxx.bbb >= yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= yyy.f_low and >= and < yyy.bbb and xxx.bbb < yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= yyy.f_low and < and xxx.bbb >= and xxx.bbb >= yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= yyy.f_low and < and xxx.bbb >= and xxx.bbb < yyy.bbb
and xxx.f_low < yyy.f_high and xxx.f_high >= yyy.f_low and xxx.f_low >= yyy.f_low and < and xxx.bbb <


So what can we do with these ?

We have xxx and yyy because we have 3 periods to combine in the relational process, today, yesterday and the day before yesterday - that is for the relationship between today and yesterday xxx and yyy will be today and yesterday, and the relationship between yesterday and the day before, xxx and yyy will be yesterday and the day before - aaa and bbb are the dependent on if close is higher than open, but are either open or close prices.

What we can say is that for today and yesterday we can take the first statement and calculate the success coefficient (how well it conforms to wealth increase) and then we can combine with all of these same statements for the periods 'yesterday and the day before', again to give separate success coefficients, and then run the statements with the highest success coefficients on a daily basis, if triggered then tweet and /or combine with the current traces for further analysis

The statements are also designed around database sql statements and so plug directly into the Candlestick trace engine and hence the 'and' at the start of each sentence

This is purely for the relational aspect of Candlesticks, we also need to add the intra period and trend formulae on top of this

Friday, 15 June 2012

candlesticks redux

Whilst the traces are progressing, I have focused on the processing of Candlesticks.

With an eye on what is assuredly a method that has gone through a lot of development over many years, I have re-engineered and carried out a data analysis on Candlesticks. There appears to be 3 aspects that are relevant:

1. Relational
This is the relationship between periods and states things like 'high today is higher than the high yesterday'

2. Elemental
This is within the period and states things like 'the close is very close to the high and the (close - open) is 0.005 of (high - low) 

3. Trend
Allocating trends on top of the previous 1 and 2, ie saying things like 'on a bullish trend and if todays high is greater than yesterday's high'

If we take all possible combinations of the above we get approximately 43 factorial different combinations of possible formulae. This is 6 to the power 52 - and wrong. It is wrong because we end up with statements like 'if todays high is less than todays low' - so we need to eliminate all combinations that do not make sense. This exercise for the Relational section gives us 22 different options, Elemental is 3, and Trend is 2.

The permutations here now (rather than combinations as per the traces), gives 22 x 22 x 4 x 3 different correct permutations: 5,808 - the 4 and the 3 are one plus Elemental and Trend - and compared to the 300,000 odd combinations being currently calculated for each company via the traces.

Also needing to be taken into consideration is if the close for a period (within each period) is higher or lower than the period's open - there being 4 options - the options of current and previous period close and open - but still 5,808 options per combination.

What these 5,808 permutations give us is every single possible combination of states between 3 periods and prices of open, close, high and low.

And so the Candlestick trace engine is taking shape. When completed and run we will have success indicators for every single type of Candlestick combination for every company. We can then match these Candlestick traces to current period indicators, giving us the same type of information as the traces.

What I am expecting to see is that the combinations with the best success coefficients will have been named. I have done this re-engineering because the first pass at Candlestick analysis has given some random results and I need to know that I have covered all bases, in this case all possible permutations - it also makes certain that I havent missed anything.

Sunday, 10 June 2012


GBP on the LSE is in, BVC, BDEV, EMIS and IPO are in and repeated. AMER is now out of the trace formulae as is OPTS.

Having done analysis on the traces, what is happening is that different formulae are being triggered on subsequent days, giving a range of days that companies follow triggers. This reinforces the notion that all the formulae that state when a stock is supposed to rise, when added together, give no companies - and follows that the number and percentages for companies being returned for formulae that contain more than 7 elements drops dramatically.

What does this mean ? It means I will be watching all the above companies over the next 5 days.

The number of companies and trace records currently are -  LSE:491:26m, NYSE:144:2.7m, NASDAQ:257:1.7m - getting there slowly

Thursday, 7 June 2012

EMIS, BVC, BDEV, AMER and something called AUP3

The above triggered today, AMER is third day running but as we know we have already had a 28% increase on the first day so interesting to know what happens tomorrow, now watching EMIS, BVC and BDEV. AUP3 is a peculiar one and is 'ETFS FOREIGN EXCHANGE LIMITED ETFS 3X LONG AUD SHORT GBP' - does this mean anything to anyone ? cos by all accounts it should go up in the next 5 days by 11.5%.

Not sure you have noticed but the trace calculations for all exchanges is carrying on 24 hours a day, one company about every 2 hours, 3 exchanges in parallel, and I am only up to about the A's on LSE, NYSE and B's on the NASDAQ. At the start I had been testing by tracing random companies and so the reason why we are getting companies starting with different letters for the LSE. It also seems that the traced companies for the NYSE and NASDAQ have not hit any companies that trigger on the specific day as yet (last three days). I am hoping this changes as more companies are traced, in fact I am quite surprised at the number being returned by the LSE traces, the auto tweets tweet the top 5 results for LSE_trace; today oasis returned 7, the missing ones being OPTS (again) and IPO (again), these repeats seem also to be interesting and I am keeping a close watch on them. We almost have a full set of companies from all exchanges that have been traced and start with 'A'... at least its a start, and trading only companies that start with 'A's seems just as limiting as trading only oilies, or minies.
Also, for information, the number of trace records currently for LSE after 451 companies is 24,069,113; this is about a fifth of the way through LSE. I am expecting a similar total number for NYSE and NASDAQ; NYSE currently stands at 1,055,174 records for 59 companies traced and for the NASDAQ, 700,837 records for 111 companies traced.

Also, by my calculations the complete trace for the three exchanges is going to take about 4 months to complete running 24 hours a day on my system, and my system is not small. Anyone with a Exadata or similar want to share resources ?