Evaluation Criterion: Validity

When the time to conduct research has come, we would deal with the following questions: Has the research method been accurate? Are the data we gained valid? Are questions that we used in the questionnaire reliable?

We must be able to answer those questions to ensure that our research study has followed the principles of scientific research. But, do you know what is meant by a valid and reliable data? If you still have no idea, I think this material would be perfect for you..

When we are formulating a measurement scale, i.e. the questions in the questionnaire, we have to evaluate whether the scale has been followed accurately in accordance with the principles of scientific research. In general, there are three conditions in which the questions we use are accurate: valid, has good reliability, and the can generalize to other situation. At this time, we will discuss the first criterion in research evaluation that is validity.

Practically, validity is there in every phase in research process, starting from planning process even to reporting. Before we input and process the data, we need to check the validity of instrument we used and the sample we took. The test of instrument validity includes evaluation on research samples, i.e. whether the selected sample has been valid or can represent the population closely, and evaluating on measurement tools used in research. We can conduct a pre-test to check whether our instrument (both sample and questions) are in the level of validity that can be tolerant for scientific research. While, evaluation on data validity is conducted before we input the data. We will need any statistical calculations such as t-value, standard deviation, etc. You can use statistical software like SPSS, LISREL, STATA and so on.

Definition of Validity

Data said to be valid if it is free from systematic bias and random error. Researchers define bias as the tendency of the facts obtained from respondents (or data) is influenced by other factors not included in the defined variables. Random error means the data are influenced by other factors, but in a random pattern, does not interest in one particular direction. In a simple validity definition, valid is a condition where data gained from the sample can accurately describe the real condition of measured population. Perfect validity means there is no significant difference between the mean value of the sample with mean value of a population parameter (XO = XT, XR = 0, XS = 0). And, valid argument can be said that argument is based on real facts.

Generally, there are three types of validity in research: content validity, criterion validity, and construct validity.

Content Validity

When we examine the content validity, actually what we do is evaluating whether the measurement tools that we use (in this case the research questions) are quite comprehensive covering all the dimensions of the variables studied. For example, we want to examine the store image by using questionnaire, the dimension that needs to be studied include the quality of the goods sold, variety and completeness of goods, store atmosphere, and so on. The more complete the dimensions we use, the content validity is getting better.

Criterion Validity

Criterion validity describes the extent to which a measurement tools that we use in research work according to our expectations in measuring the relationship between variables being investigated as a meaningful criterion. Variable criteria typically includes demographic and psychographic characteristics, measurement of attitudes and behaviors, or values obtained from the scale. Based on the period, the criteria validity consists of concurrent validity and predictive validity.

Concurrent validity is assessed when the data on the scale being evaluated and on the criterion variables are collected at the same time. To assess concurent validity, we may build two version of instrument, the original and the short version, then we administer both of them simultaneously to a group of respondents and the results compared. In other side, predictive validity is assessed by collecting data on the scale at one point in time and data on the criterion variables at a future time. In predictive validity the data obtained from the measurement could be used to predict a future event. For example, attitude toward instant noodle brands could be used to predict future buying behavior of instant noodle by members of certain population.

Construct Validity

Construct validity (construct validity) refers to the question: What constructs are exactly being studied or measured? When measuring the construct validity, researchers try to answer theoretical questions why the scale is used and what conclusions can be made by considering the existing theories. Construct validity is the most difficult evaluation in research because we need enough qualified theories.

Construct validity includes convergent validity, discriminant validity, and nomological validity. Convergent validity measures the extent to which the scale was positively correlated with other measures of the same construct. Convergent validity was sought to confirm the relationship between the construct in accordance with the underlying theory. Discriminant validity evaluates the extent to which measurements are not correlated with other constructs from which it should be different. And, nomological validity is the extent to which the scale correlates in theoretically predicted ways with measures of different but related constructs. A theoretical model is formulated that leads to further deductions, tests, and inferences. This nomological net should be able to interrelate systematically those constructs.

Internal Validity and External Validity

Internal validity refers to the extent to which the observed independent variables can really affect on the dependent variables. Internal validity rule says that we must use variables that are most likely to be the main cause on dependent variables, or the main cause of certain event could happen, so that other unpredicted external factors can significantly affect the dependent variables. However, in fact, internal validity is often influenced by several factors, such as: the incident between pre-test and post-test, the age of respondents, sampling methods, identification of independent and dependent variables are still vague, and the measurement tools.

Instead of internal validity refers to the ability of the manipulated independent variables can actually affect the dependent variables, the external validity aims to see whether the cause-and-effect relationships that we found in the research can be generalized to other situation, or it means that the findings can be applied even if the conditions change, both environmental conditions and the sample used. In doing research there are three forms of validation. Validation definition is a procedure to check the degree of validity particural data, sample or research instrument.

1. Fieldwork validation. It is an attempt to examine whether the fieldworker has done his job properly. Usually, supervisor contacted 10 to 25 percent of respondents to check if the interviewer has finished the interview well, how the quality of the interview, what is respondents' opinion about the interview process, and so on.


2. Sample validation. It is to check or validate whether the sample we choose are appropriate for research. The sample should have similar characteristic with the population being observed.


3. Cross validation. If we use regression analysis method we will face this type of validation. Cross validation is a validity test to examine whether the regression model continues to hold on comparable data not used in the estimation.

Validity is an absolute criterion to evaluate research findings. The invalid findings obviously cannot be used to develop a correct conclusion because they don't represent the true fact. To obtain valid findings, we need to test the validity starting from making measurements to research, select sample, process the data, to create reports.