Martingle

History

Originally, martingale referred to a class of betting strategies popular in 18th century France. The simplest of these strategies was designed for a game in which the gambler wins his stake if a coin comes up heads and loses it if the coin comes up tails. The strategy had the gambler double his bet after every loss, so that the first win would recover all previous losses plus win a profit equal to the original stake. Since as a gambler's wealth and available time jointly approach infinity his probability of eventually flipping heads approaches 1, the martingale betting strategy was seen as a sure thing by those who practiced it. Of course in reality the exponential growth of the bets would eventually bankrupt those foolish enough to use the martingale for a long time.

The concept of martingale in probability theory was introduced by Paul Pierre Lévy, and much of the original development of the theory was done by Joseph Leo Doob. Part of the motivation for that work was to show the impossibility of successful betting strategies.

Definitions

A discrete-time martingale is a discrete-time stochastic process (i.e., a sequence of random variables) X1, X2, X3, ... that satisfies for all n

E(|Xn|) < infinity

E(Xn+1|X1,...,Xn)= Xn,

i.e., the conditional expected value of the next observation, given all of the past observations, is equal to the last observation.

Somewhat more generally, a sequence Y1, Y2, Y3, ... is said to be a martingale with respect to another sequence X1, X2, X3, ... if for all n

E(|Yn|)< infinity

E(Yn+1|X1,.....,Xn) = Yn.

For more details, Please click MARTINGLE

Thanks,

SAS Output : Proc reg

SAS Annotated Output
Regression Analysis


This page shows an example regression analysis with footnotes explaining the output. These data were collected on 200 high schools students and are scores on various tests, including science, math, reading and social studies (socst). The variable female is a dichotomous variable coded 1 if the student was female and 0 if male.

In the code below, the data = option on the proc reg statement tells SAS where to find the SAS data set to be used in the analysis. On the model statement, we specify the regression model that we want to run, with the dependent variable (in this case, science) on the left of the equals sign, and the independent variables on the right-hand side. We use the clb option after the slash on the model statement to get the 95% confidence limits of the parameter estimates. The quit statement is included because proc reg is an interactive procedure, and quit tells SAS that not to expect another proc reg immediately.

proc reg data = "d:\dataset";
model science = math female socst read / clb;
run;
quit;




SAS Annotated Output
Regression Analysis


The REG Procedure
Model: MODEL1
Dependent Variable: science science score

Analysis of Variance

Sum of Mean
Source DF Squares Square F Value Pr > F

Model 4 9543.72074 2385.93019 46.69 <.0001
Error 195 9963.77926 51.09630
Corrected Total 199 19507

Root MSE 7.14817 R-Square 0.4892
Dependent Mean 51.85000 Adj R-Sq 0.4788
Coeff Var 13.78624

Parameter Estimates

Parameter Standard
Variable Label DF Estimate Error t Value Pr > |t|

Intercept Intercept 1 12.32529 3.19356 3.86 0.0002
math math score 1 0.38931 0.07412 5.25 <.0001
female 1 -2.00976 1.02272 -1.97 0.0508
socst social studies score 1 0.04984 0.06223 0.80 0.4241
read reading score 1 0.33530 0.07278 4.61 <.0001

Parameter Estimates

Variable Label DF 95% Confidence Limits

Intercept Intercept 1 6.02694 18.62364
math math score 1 0.24312 0.53550
female 1 -4.02677 0.00724
socst social studies score 1 -0.07289 0.17258
read reading score 1 0.19177 0.47883

Anova Table

Analysis of Variance
Sum of Mean
Sourcea DFb Squaresc Squared F Valuee Pr > Fe
Model 4 9543.72074 2385.93019 46.69 <.0001
Error 195 9963.77926 51.09630
Corrected Total 199 19507

a. Source - This is the source of variance, Model, Residual, and Total. The Total variance is partitioned into the variance which can be explained by the independent variables (Model) and the variance which is not explained by the independent variables (Residual, sometimes called Error). Note that the Sums of Squares for the Model and Residual add up to the Total Variance, reflecting the fact that the Total Variance is partitioned into Model and Residual variance.

b. DF - These are the degrees of freedom associated with the sources of variance. The total variance has N-1 degrees of freedom. In this case, there were N=200 students, so the DF for total is 199. The model degrees of freedom corresponds to the number of predictors minus 1 (K-1). You may think this would be 4-1 (since there were 4 independent variables in the model, math, female, socst and read). But, the intercept is automatically included in the model (unless you explicitly omit the intercept). Including the intercept, there are 5 predictors, so the model has 5-1=4 degrees of freedom. The Residual degrees of freedom is the DF total minus the DF model, 199 - 4 is 195.

c. Sum of Squares - These are the Sum of Squares associated with the three sources of variance, Total, Model and Residual. These can be computed in many ways. Conceptually, these formulas can be expressed as:
SSTotal The total variability around the mean. S(Y - Ybar)2.
SSResidual The sum of squared errors in prediction. S(Y - Ypredicted)2.
SSModel The improvement in prediction by using the predicted value of Y over just using the mean of Y. Hence, this would be the squared differences between the predicted value of Y and the mean of Y, S(Ypredicted - Ybar)2. Another way to think of this is the SSModel is SSTotal - SSResidual. Note that the SSTotal = SSModel + SSResidual. Note that SSModel / SSTotal is equal to .4892, the value of R-Square. This is because R-Square is the proportion of the variance explained by the independent variables, hence can be computed by SSModel / SSTotal.

d. Mean Square - These are the Mean Squares, the Sum of Squares divided by their respective DF. For the Model, 9543.72074 / 4 = 2385.93019. For the Residual, 9963.77926 / 195 = 51.0963039. These are computed so you can compute the F ratio, dividing the Mean Square Model by the Mean Square Residual to test the significance of the predictors in the model.

e. F Value and Pr > F - The F-value is the Mean Square Model (2385.93019) divided by the Mean Square Residual (51.0963039), yielding F=46.69. The p-value associated with this F value is very small (0.0000). These values are used to answer the question "Do the independent variables reliably predict the dependent variable?". The p-value is compared to your alpha level (typically 0.05) and, if smaller, you can conclude "Yes, the independent variables reliably predict the dependent variable". You could say that the group of variables math, female, socst and read can be used to reliably predict science (the dependent variable). If the p-value were greater than 0.05, you would say that the group of independent variables does not show a statistically significant relationship with the dependent variable, or that the group of independent variables does not reliably predict the dependent variable. Note that this is an overall significance test assessing whether the group of independent variables when used together reliably predict the dependent variable, and does not address the ability of any of the particular independent variables to predict the dependent variable. The ability of each individual independent variable to predict the dependent variable is addressed in the table below where each of the individual variables are listed.
Overall Model Fit

Root MSEf 7.14817 R-Squarei 0.4892
Dependent Meang 51.85000 Adj R-Sqj 0.4788
Coeff Varh 13.78624

f. Root MSE - Root MSE is the standard deviation of the error term, and is the square root of the Mean Square Residual (or Error).

g. Dependent Mean - This is the mean of the dependent variable.

h. Coeff Var - This is the coefficient of variation, which is a unit-less measure of variation in the data. It is the root MSE divided by the mean of the dependent variable (7.15/51.85 = 13.79).

i. R-Square - R-Square is the proportion of variance in the dependent variable (science) which can be predicted from the independent variables (math, female, socst and read). This value indicates that 48.92% of the variance in science scores can be predicted from the variables math, female, socst and read. Note that this is an overall measure of the strength of association, and does not reflect the extent to which any particular independent variable is associated with the dependent variable.

j. Adj R-Sq - Adjusted R-square. As predictors are added to the model, each predictor will explain some of the variance in the dependent variable simply due to chance. One could continue to add predictors to the model which would continue to improve the ability of the predictors to explain the dependent variable, although some of this increase in R-square would be simply due to chance variation in that particular sample. The adjusted R-square attempts to yield a more honest value to estimate the R-squared for the population. The value of R-square was .4892, while the value of Adjusted R-square was .4788 Adjusted R-squared is computed using the formula 1 - ((1 - Rsq)((N - 1) / (N - k - 1)). From this formula, you can see that when the number of observations is small and the number of predictors is large, there will be a much greater difference between R-square and adjusted R-square (because the ratio of (N - 1) / N - k - 1) will be much less than 1). By contrast, when the number of observations is very large compared to the number of predictors, the value of R-square and adjusted R-square will be much closer because the ratio of (N - 1)/(N - k - 1) will approach 1.
Parameter Estimates

Parameter Estimates

Parameter Standard
Variablek Labell DFm Estimaten Erroro t Valuep Pr > |t|p

Intercept Intercept 1 12.32529 3.19356 3.86 0.0002
math math score 1 0.38931 0.07412 5.25 <.0001
female 1 -2.00976 1.02272 -1.97 0.0508
socst social studies score 1 0.04984 0.06223 0.80 0.4241
read reading score 1 0.33530 0.07278 4.61 <.0001

Parameter Estimates

Variablek Labell DFm 95% Confidence Limitsq

Intercept Intercept 1 6.02694 18.62364
math math score 1 0.24312 0.53550
female 1 -4.02677 0.00724
socst social studies score 1 -0.07289 0.17258
read reading score 1 0.19177 0.47883

k. Variable - This column shows the predictor variables (constant, math, female, socst, read). The first variable (constant) represents the constant, also referred to in textbooks as the Y intercept, the height of the regression line when it crosses the Y axis. In other words, this is the predicted value of science when all other variables are 0.

l. Label - This column gives the label for the variable. Usually, variable labels are added when the data set is created so that it is clear what the variable is (as the name of the variable can sometimes be ambiguous). SAS has labeled the variable Intercept for us by default. Note that this variable is not added to the data set.

m. DF - This column give the degrees of freedom associated with each independent variable. All continuous variables have one degree of freedom, as do binary variables (such as female).

n. Parameter Estimates - These are the values for the regression equation for predicting the dependent variable from the independent variable. The regression equation is presented in many different ways, for example:

Ypredicted = b0 + b1*x1 + b2*x2 + b3*x3 + b4*x4

The column of estimates (coefficients or parameter estimates, from here on labeled coefficients) provides the values for b0, b1, b2, b3 and b4 for this equation. Expressed in terms of the variables used in this example, the regression equation is

sciencePredicted = 12.32529 + .3893102*math + -2.009765*female+.0498443*socst+.3352998*read

These estimates tell you about the relationship between the independent variables and the dependent variable. These estimates tell the amount of increase in science scores that would be predicted by a 1 unit increase in the predictor. Note: For the independent variables which are not significant, the coefficients are not significantly different from 0, which should be taken into account when interpreting the coefficients. (See the columns with the t-value and p-value about testing whether the coefficients are significant).
math - The coefficient (parameter estimate) is .3893102. So, for every unit (i.e., point, since this is the metric in which the tests are measured) increase in math, a .3893102 unit increase in science is predicted, holding all other variables constant. (It does not matter at what value you hold the other variables constant, because it is a linear model.) Or, for every increase of one point on the math test, your science score is predicted to be higher by .3893102 points. This is significantly different from 0.
female - For every unit increase in female, there is a -2.009765 unit decrease in the predicted science score, holding all other variables constant. Since female is coded 0/1 (0=male, 1=female) the interpretation can be put more simply. For females the predicted science score would be 2 points lower than for males. The variable female is technically not statistically significantly different from 0, because the p-value is greater than .05. However, .051 is so close to .05 that some researchers would still consider it to be statistically significant.
socst - The coefficient for socst is .0498443. This means that for a 1-unit increase in the social studies score, we expect an approximately .05 point increase in the science score. This is not statistically significant; in other words, .0498443 is not different from 0.
read - The coefficient for read is .3352998. Hence, for every unit increase in reading score we expect a .34 point increase in the science score. This is statistically significant.

o. Standard Error - These are the standard errors associated with the coefficients. The standard error is used for testing whether the parameter is significantly different from 0 by dividing the parameter estimate by the standard error to obtain a t-value (see the column with t-values and p-values). The standard errors can also be used to form a confidence interval for the parameter, as shown in the last two columns of this table.

p. t Value and Pr > |t|- These columns provide the t-value and 2 tailed p-value used in testing the null hypothesis that the coefficient/parameter is 0. If you use a 2 tailed test, then you would compare each p-value to your preselected value of alpha. Coefficients having p-values less than alpha are statistically significant. For example, if you chose alpha to be 0.05, coefficients having a p-value of 0.05 or less would be statistically significant (i.e., you can reject the null hypothesis and say that the coefficient is significantly different from 0). If you use a 1 tailed test (i.e., you predict that the parameter will go in a particular direction), then you can divide the p-value by 2 before comparing it to your preselected alpha level. With a 2-tailed test and alpha of 0.05, you may reject the null hypothesis that the coefficient for female is equal to 0. The coefficient of -2.009765 is significantly greater than 0. However, if you used a 2-tailed test and alpha of 0.01, the p-value of .0255 is greater than 0.01 and the coefficient for female would not be significant at the 0.01 level. Had you predicted that this coefficient would be positive (i.e., a one tail test), you would be able to divide the p-value by 2 before comparing it to alpha. This would yield a one-tailed p-value of 0.00945, which is less than 0.01 and then you could conclude that this coefficient is greater than 0 with a one tailed alpha of 0.01.
The coefficient for math is significantly different from 0 using alpha of 0.05 because its p-value is 0.000, which is smaller than 0.05.
The coefficient for socst (0.0498443) is not statistically significantly different from 0 because its p-value is definitely larger than 0.05.
The coefficient for read (0.3353) is statistically significant because its p-value of 0.000 is less than .05.
The constant (_cons) is significantly different from 0 at the 0.05 alpha level. However, having a significant intercept is seldom interesting.

q. 95% Confidence Limits - This shows a 95% confidence interval for the coefficient. This is very useful as it helps you understand how high and how low the actual population value of the parameter might be. The confidence intervals are related to the p-values such that the coefficient will not be statistically significant if the confidence interval includes 0. If you look at the confidence interval for female, you will see that it just includes 0 (-4 to .007). Because .007 is so close to 0, the p-value is close to .05. If the upper confidence level had been a little smaller, such that it did not include 0, the coefficient for female would have been statistically significant. Also, consider the coefficients for female (-2) and read (.34). Immediately you see that the estimate for female is so much bigger, but examine the confidence interval for it (-4 to .007). Now examine the confidence interval for read (.19 to .48). Even though female has a bigger coefficient (in absolute terms) it could be as small as -4. By contrast, the lower confidence level for read is .19, which is still above 0. So, even though female has a bigger coefficient, read is significant and even the smallest value in the confidence interval is still higher than 0. The same cannot be said about the coefficient for socst. Such confidence intervals help you to put the estimate from the coefficient into perspective by seeing how much the value could vary.

SAS : Proc Reg

Proc Reg : Regression procedure

The REG procedure fits least-squares estimates to linear regression models. The following statements are used with the REG procedure:

PROC REG options;
MODEL dependents=regressors / options;
VAR variables;
FREQ variable;
WEIGHT variable;
ID variable;
OUTPUT OUT=SASdataset keyword=names...;
PLOT yvariable*xvariable = symbol ...;
RESTRICT linear_equation,...;
TEST linear_equation,...;
MTEST linear_equation,...;
BY variables;

The PROC REG statement is always accompanied by one or more MODEL statements to specify regression models. One OUTPUT statement may follow each MODEL statement. Several RESTRICT, TEST, and MTEST statements may follow each MODEL. WEIGHT, FREQ, and ID statements are optionally specified once for the entire PROC step. The purposes of the statements are:

* The MODEL statement specifies the dependent and independent variables in the regression model.
* The OUTPUT statement requests an output data set and names the variables to contain predicted values, residuals, and other output values.
* The ID statement names a variable to identify observations in the printout.
* The WEIGHT and FREQ statements declare variables to weight observations.
* The BY statement specifies variables to define subgroups for the analysis. The analysis is repeated for each value of the BY variable.

Proc REG Statement

PROC REG options;

These options may be specified on the PROC REG statement:

DATA=SASdataset
names the SAS data set to be used by PROC REG. If DATA= is not specified, REG uses the most recently created SAS data set.
OUTEST=SASdataset
requests that parameter estimates be output to this data set.
OUTSSCP=SASdataset
requests that the crossproducts matrix be output to this TYPE=SSCP data set.
NOPRINT
suppresses the normal printed output.
SIMPLE
prints the "simple" descriptive statistics for each variable used in REG.
ALL
requests many different printouts.
COVOUT
outputs the covariance matrices for the parameter estimates to the OUTEST data set. This option is valid only if OUTEST= is also specified.

MODEL Statement

label: MODEL dependents = regressors / options;

After the keyword MODEL, the dependent (response) variables are specified, followed by an equal sign and the regressor variables. Variables specified in the MODEL statement must be variables in the data set being analyzed. The label is optional.

* General options:

NOPRINT
suppresses the normal printout of regression results.
NOINT
suppresses the intercept term that is normally included in the model automatically.
ALL
requests all the features of these options: XPX, SS1, SS2, STB, TOL, COVB, CORRB, SEQB, P, R, CLI, CLM.

* Options to request regression calculations:

XPX
prints the X'X crossproducts matrix for the model.
I
prints the (X'X)-1 matrix.

* Options for details on the estimates:

SS1
prints the sequential sums of squares (Type I SS) along with the parameter estimates for each term in the model.
SS2
prints the partial sums of squares (Type II SS) along with the parameter estimates for each term in the model.
STB
prints standardized regression coefficients.
TOL
prints tolerance values for the estimates.
VIF
prints variance inflation factors with the parameter estimates. Variance inflation is the reciprocal of tolerance.
COVB
prints the estimated covariance matrix of the estimates.
CORRB
prints the correlation matrix of the estimates.
SEQB
prints a sequence of parameter estimates as each variable is entered into the model.
COLLIN
requests a detailed analysis of collinearity among the regressors.
COLLINOINT
requests the same analysis as the COLLIN option with the intercept variable adjusted out rather than included in the diagnostics.

* Options for predicted values and residuals:

P
calculates predicted values from the input data and the estimated model.
R
requests that the residual be analyzed.
CLM
prints the 95% upper and lower confidence limits for the expected value of the dependent variable (mean) for each observation.
CLI
requests the 95% upper and lower confidence limits for an individual predicted value.
DW
calculates a Durbin-Watson statistic to test whether or not the errors have first-order autocorrelation. (This test is only appropriate for time-series data.)
INFLUENCE
requests a detailed analysis of the influence of each observation on the estimates and the predicted values.
PARTIAL
requests partial regression leverage plots for each regressor.

FREQ Statement

FREQ variable;

If a variable in your data set represents the frequency of occurrence for the other values in the observation, include the variable's name in a FREQ statement. The procedure then treats the data set as if each observation appears n times, where n is the value of the FREQ variable for the observation. The total number of observations will be considered equal to the sum of the FREQ variable when the procedure determines degrees of freedom for significance probabilities.
WEIGHT Statement

WEIGHT variable;

A WEIGHT statement names a variable on the input data set whose values are relative weights for a weighted least-squares fit. If the weight value is proportional to the reciprocal of the variance for each observation, then the weighted estimates are the best linear unbiased estimates (BLUE).
ID Statement

ID variable;

The ID statement specifies one variable to identify observations as output from the MODEL options P, R, CLM, CLI, and INFLUENCE.
OUTPUT Statement

The OUTPUT statement specifies an output data set to contain statistics calculated for each observation. For each statistic, specify the keyword, an equal sign, and a variable name for the statistic on the output data set. If the MODEL has several dependent variables, then a list of output variable names can be specified after each keyword to correspond to the list of dependent variables.

OUTPUT OUT=SASdataset
PREDICTED=names or P=names
RESIDUAL=names or R=names
L95M=names
U95M=names
L95=names
U95=names
STDP=names
STDR=names
STUDENT=names
COOKD=names
H=names
PRESS=names
RSTUDENT=names
DFFITS=names
COVRATIO=names;

The output data set named with OUT= contains all the variables for which the analysis was performed, including any BY variables, any ID variables, and variables named in the OUTPUT statement that contain statistics.

These statistics may be output to the new data set:

PREDICTED=
P=
predicted values.
RESIDUAL=
R=
residuals, calculated as ACTUAL minus PREDICTED.
L95M=
lower bound of a 95% confidence interval for the expected value (mean) of the dependent variable.
U95M=
upper bound of a 95% confidence interval for the expected value (mean) of the dependent variable.
L95=
lower bound of a 95% confidence interval for an individual prediction. This includes the variance of the error as well as the variance of the parameter estimates.
U95=
upper bound of a 95% confidence interval for an individual prediction.
STDP=
standard error of the mean predicted value.
STDR=
standard error of the residual.
STUDENT=
studentized residuals, the residual divided by its standard error.
COOKD=
Cook's D influence statistic.
H=
leverage.
PRESS=
residual for estimates dropping this observation, which is the residual divided by (1-h) where h is leverage above.
RSTUDENT=
studentized residual defined slightly differently than above.
DFFITS=
standard influence of observation on predicted value.
COVRATIO=
standard influence of observation on covariance of betas, as discussed with INFLUENCE option.

PLOT Statement

PLOT yvariable*xvariable=symbol / options

The PLOT statement prints scatter plots of the yvariables on the vertical axis and xvariables on the horizontal axis. It uses the symbol specified to mark the points. The yvariables and xvariables may be any variables in the data set or any of the calculated statistics available in the OUTPUT statement.
TEST Statement

label: TEST equation1,
equation2,
.
.
.
equationk;
label: TEST equation1,..., equationk / options;

The TEST statement, which has the same syntax as the RESTRICT statement except for options, tests hypotheses about the parameters estimated in the preceding MODEL statement. Each equation specifies a linear hypothesis to be tested.

One option may be specified in the TEST statement after a slash (/):

PRINT
prints intermediate calculations.

BY Statement

BY variables;

A BY statement may be used with PROC REG to obtain separate analyses on observations in groups defined by the BY variables. When a BY statement appears, the procedure expects the input data set to be sorted in order of the BY variables. If your input data set is not sorted in ascending order, use the SORT procedure with a similar BY statement to sort the data, or, if appropriate, use the BY statement options NOTSORTED or DESCENDING.