Details     

 

 

 Home

 

 

Click Here to Contact Dan Andresen

 

 

 


The purpose of this webpage is to describe in detail a math question that we are attempting to solve. Below we will describe the question as well as some required parameters that must be adhered to.

The Bottom Line to our problem is :  We are trying to take a simple, un-weighted moving average and
lower that average by distributing different weights to different data points along the way.

 Our problem is almost perfectly described in what is known as the Secretary Problem (and/or other, similar Optimal Stopping problems).

 


DETAILS OF THE QUESTION


 

The following is an detailed explanation of the math question we have:

 

1) We have a long series of computer generated numbers or "data points"  (over 20,000 data points displayed in this sample).

  (Click on thumbnail to enlarge)

 

2) We take that long series of numbers and split it into groups of 10 data points.

  (Click on thumbnail to enlarge)


 

3)   This is the first Group of 10 Data Points.  This first Group will be used in the explanations below.

 

 

 

 

4) The first Group of 10 Data Points has an Average value of 7330.6

 

 

5)  In the picture below, the Group's average of 7330.6 is being calculated at each Data Point. 

The Average uses a "Weight" of 1 on each Data Point (shown in the Green "1") creating an Un-weighted Average.

By the 10th data Point, the average of all 10 Data Points is 7330.6

 

 

6)  This chart shows that by applying DIFFERENT WEIGHTS to each Data Point, you can manipulate the Weighted Average to be lower. 

The Red "Weights" vary from 0 (on the 2nd Data Point) to 3 (on the 5th Data Point), creating a Weighted Average.

By the 10th data Point, the average of all 10 Data Points is 7326.4 (which is lower than the Un-weighted Average of 7330.6 by 3.8 points).

 

Our goal is to find a way to calculate the AMOUNT OF WEIGHTS to be placed at any given data point in order
to achieve a lower Overall Weighted Average, as displayed by the Red-colored Weights and Red-colored Weighted Average shown in the graph above. 
What we need is a mathematical formula to help us determine how much weight to apply to each data point.

 

 

 

 


 


ADDITIONAL REQUIREMENTS
 


 

Below are some requirements that the formula must take in to account.

A) These numbers are generated 1 at a time. You do not know what any of the future numbers are.
You DO know about all the past numbers (including all the numbers from the previous data sets) and you can use that information to help determine future likelihoods, estimates and probabilities.

B) You can place any number of weights on any Data Point, but you must have placed all 10 Weights on or by the 10th and final Data Point. 

C) The ultimate goal is for the Overall Weighted Average to be lower than the Overall Un-weighted Average by at least 1 point.
Obviously, all the data sets will not be able to achieve this objective, but the goal is to have the Overall Weighted Average less than 1 full point below the Overall Un-weighted Average.

 

 

 


WHAT WE NEED
 


 

We need a mathematical formula to determine how much Weight to apply at any given Data Point, with the ultimate goal of this formula producing results such that: The Overall Weighted Average must be 1 full point less than the Overall Un-weighted Average.

 

 

 

 

 


sample of hypothetical results data


 

The Purpose of this Sample Hypothetical Results Example is to show that the Results of the Math Formula are deemed "successful" or not (i.e. was the Overall Weighted Average 1 full point less than the Overall Un-weighted Average).  This table below shows a format in which you can display your Output/Results.

 

Column A shows the Un-weighted Average of each Group of 10 Data Points.

Column B is where a Math Formula was used to place different weightings on each of the 10 Data Points with the result being a Weighted Average.

Column C is the difference between the Un-weighted Average and the Math Formula's Weighted Average. 


 

             
    Column A Column B   Column C  
             
      Math   In This Group  
      Formula's   Weighted  
Group of 10   Un-weighted Weighted   Average  
Data Points   Average Average   is Lower By  
             
1st Group of 10 Data Points   7330.6 7328.1   2.5  
2nd Group of 10 Data Points   7335.9 7333.8   2.1  
3rd Group of 10 Data Points   7327.9 7331.1   -3.2  
4th Group of 10 Data Points   7319 7317.2   1.8  
5th Group of 10 Data Points   7319.5 7317.2   2.3  
6th Group of 10 Data Points   7315.4 7313.7   1.7  
7th Group of 10 Data Points   7308.9 7311   -2.1  
8th Group of 10 Data Points   7310.5 7307.3   3.2  
9th Group of 10 Data Points   7319 7316.9   2.1  
10th Group of 10 Data Points   7323.7 7326.9   -3.2  
11th Group of 10 Data Points   7316.7 7314.9   1.8  
12th Group of 10 Data Points   7309.1 7306.8   2.3  
13th Group of 10 Data Points   7323.2 7321.5   1.7  
14th Group of 10 Data Points   7321.6 7323.4   -1.8  
15th Group of 10 Data Points   7326.3 7324.2   2.1  
16th Group of 10 Data Points   7316.8 7313.6   3.2  
17th Group of 10 Data Points   7333.4 7331.6   1.8  
18th Group of 10 Data Points   7343.3 7345.6   -2.3  
19th Group of 10 Data Points   7342.8 7341.1   1.7  
20th Group of 10 Data Points   7313.9 7312.1   1.8  
21st Group of 10 Data Points   7301.8 7299.5   2.3  
22nd Group of 10 Data Points   7314.6 7316.3   -1.7  
23rd Group of 10 Data Points   7315.5 7312.3   3.2  
24th Group of 10 Data Points   7312.6 7311.2   1.4  
25th Group of 10 Data Points   7306.3 7308   -1.7  
26th Group of 10 Data Points   7285.1 7283   2.1  
27th Group of 10 Data Points   7275.2 7272.7   2.5  
28th Group of 10 Data Points   7272.9 7268.8   4.1  
29th Group of 10 Data Points   7280.9 7283   -2.1  
30th Group of 10 Data Points   7293.5 7291   2.5  
31st Group of 10 Data Points   7278.9 7277.2   1.7  
             
Totals of all 31 Groups of 10 Data Points   226694.8 226661   33.8  
             
             
          Overall  
    Overall Overall   Weighted  
    Un-weighted Weighted   Average  
    Average Average   is Lower By  
             
  Overall Averages   7312.735 7311.645   1.09  
             
             

The ultimate goal is for the Overall Weighted Average to be lower than the Overall Un-weighted Average by at least 1 point, as shown in the Results Chart above.

 

 

 

 

 


ALTERNATIVE RULES


 

1)  You can use a different number of Data Points in a Group.  You may define a Group as either 5, 10 or 15 Data Points;  no more than 15 Data Points in a group is permitted.  In any case, a Group of N Data Points has to have placed a total of N "weights" by the end of the Group.  When one Group ends, the very next Data Point is the first Data Point of the next Group -- no skipping data Points in between Groups.

2)  You may choose varying lengths of Data Points to call a "Group".  For example, you may have Group of 5, then a Group of 3, then a group of 12, followed by a Group of 15, then a Group of 6.  Group length can't be more than 15 in any case.  In any case, a Group of N Data Points has to have placed a total of N "weights" by the end of the Group.  When one Group ends, the very next Data Point is the first Data Point of the next Group -- no skipping data Points in between Groups.

3)  Weights can be smaller than 1 unit.  The smallest unit is 1/2 of 1.

4)  A Weight can be a negative number as well, as long as the maximum weights placed is equal to the number of Data Points and by the final Data Point, you have placed all N weights.  For example, if you have 10 as your number of Data Points, at the 10th Data Point, you have to have placed all 10 weights.  Also, at any given point along the way you can placed as many weights as you want, but you can't be more than +10 or -10 overall at any time.

Example on a 10-Data Point Group:

 

Data Point Number Weight Placed This Data Point Combined Weights Placed Comments
1 +1 +1  
2 -4 -3  
3 -6 -9  
4 -1 -10 Hit Max -10 Overall
5 +13 +3  
6 +2 +5  
7 -1 +4  
8 +6 +10 Hit Max +10 Overall
9 -2 +8  
10 +2 +10 Final Data Point, all 10 weights are placed

By the 10th Data Point... you have placed exactly 10 weights.

 

 

 

 

 


POSSIBLE FACTORS TO CONSIDER


 

Here's an example of factors that we feel would likely be useful in creating a formula to determine the number of Weights to place on any given Data Point. THESE ARE MERELY SUGGESTIONS -- THIS MAY OR MAY NOT BE THE BEST APPROACH TO THIS PROBLEM.

The chart below shows the Un-weighted Average, the Spread (distance between a given Data Point and that Un-weighted Average), and the Range of the 10-Data point set (distance from Highest Data Point to the Lowest Data Point).

Using these factors (and perhaps prior Data / Averages / Spreads / Ranges / etc), it should be possible to create a formula that can, on any given Data Point, combine this information and determine how many Weights to place ("should THIS Data Point carry EXTRA Weights or should it carry normal weighting or do the factors suggest it carry FEWER Weights").

 

 

 

 

 

 

 

 

 

 

 

 

 

              Factors that will likely be part of a formula:

Factor #1 What Turn are you on  (1 thru 10)        
Factor #2 How many pieces do I have left  (0 thru 10)        
Factor #3 Distance between the Data Point and the un-weighted Average (the "Spread")  
Factor #4  Rate of Change in the Spread from Data Point to Data Point    
Factor #5  Where are you in relation to a typical Group of 10 Data Point's Range      
                 
                 
                 
                 
   

 

 

              Here is the actual data used in the chart above.

       
Un-weighted   Rate of Change

Data Point

Average Spread of the Spread
       
7307 7307.00 0.00 0.00  
7311 7309.00 2.00 2.00  
7304 7307.33 -3.33 -5.33  
7292 7303.50 -11.50 -8.17  
7291 7301.00 -10.00 1.50  
7300 7300.83 -0.83 9.17  
7297 7300.29 -3.29 -2.45  
7299 7300.13 -1.13 2.16  
7303 7300.44 2.56 3.68  
7302 7300.60 1.40 -1.16  
       

 

 

 

 


SAMPLE DATA


 

 

Listed below are several sets of sample data. The data samples differ mainly in their "ranges" (the numbers vary to a wider degree within each set of 10, as seen in the picture below).

Data Sets #1a, 1b and 1c are the preferred sets to use.  The reason we are including the other data sets is that if the objective (of lowering the overall average down by 1 full point) can't be obtained using Data Sets #1a/b/c, then the other data sets may make it easier to accomplish the objective.

 

The following 3 Data Sets are the Preferred Data Sets to use -- they have the smallest "range" and most number of Data Points.

Click Here for Sample Data #1a File

Click Here for Sample Data #1b File 

Click Here for Sample Data #1c File

 

Here are some Alternative Data Sets:

Click Here for Sample Data #2 File  (The second best Data Set choice -- it has a slightly wider "range" and somewhat fewer number of Data Points than Data Set #1).  

Click Here for Sample Data #3 File  (The third best Data Set choice -- it has a slightly wider "range" and somewhat fewer number of Data Points than Data Set #2). 

Click Here for Sample Data #4 File  (The fourth best Data Set choice -- it has a slightly wider "range" and somewhat fewer number of Data Points than Data Set #3). 

 

There are additional samples available of each of the Data Sets if you need/want more data (just email or call us and we'll provide more).

 

 

 

 

 

 


 

Click Here to Contact Dan Andresen

 

 

 
"EasyTracking" is a trademark of Dynamic Design Software. All other products mentioned are registered trademarks or trademarks of their respective companies.

Questions, comments or suggestions for this web site should be directed to the webauthor.
Copyright © 1995-2013 Dynamic Design Software. All rights reserved. Read our Privacy Notice.
Last modified:  January 25th, 2013