A Pages:15 Words:3855
This is just a sample.
To get a unique essay
Hire Writer
Type of paper: Essay
University/College: University of Arkansas System
Subject: ,
Download: .pdf, .docx, .epub, .txt

A limited time offer!

Get custom essay sample written according to your requirements

Urgent 3h delivery guaranteed

Order Now

Demand Forecasting and Production Planning

ScienceAsia 27 (2001) : 271-278 Demand Forecasting and Production Planning for Highly Seasonal Demand Situations: Case Study of a Pressure Container Factory Pisal Yenradeea,*, Anulark Pinnoib and Amnaj Charoenthavornyingb a Industrial Engineering Program, Sirindhorn International Institute of Technology, Thammasat University, Patumtani 12121, Thailand. b Industrial Systems Engineering Program, School of Advanced Technologies, Asian Institute of Technology, P. O.

Demand Forecasting and Production Planning specifically for you
for only $13.90/page
Order Now

Box 4, Klong Luang, Patumtani 12120, Thailand. * Corresponding author, E-mail: [email protected] tu. ac. th Received 24 May 2001 Accepted 27 Jul 2001

ABSTRACT This paper addresses demand forecasting and production planning for a pressure container factory in Thailand, where the demand patterns of individual product groups are highly seasonal. Three forecasting models, namely, Winter’s, decomposition, and Auto-Regressive Integrated Moving Average (ARIMA), are applied to forecast the product demands. The results are compared with those obtained by subjective and intuitive judgements (which is the current practice). It is found that the decomposition and ARIMA models provide lower forecast errors in all product groups.

As a result, the safety stock calculated based on the errors of these two models is considerably less than that of the current practice. The forecasted demand and safety stock are subsequently used as inputs to determine the production plan that minimizes the total overtime and inventory holding costs based on a fixed workforce level and an available overtime. The production planning problem is formulated as a linear programming model whose decision variables include production quantities, inventory levels, and overtime requirements. The results reveal that the total costs could be reduced by 13. % when appropriate forecasting models are applied in place of the current practice. KEYWORDS: demand forecasting, highly seasonal demand, ARIMA method, production planning, linear programming, pressure container factory. INTRODUCTION Most manufacturing companies in developing countries determine product demand forecasts and production plans using subjective and intuitive judgments. This may be one factor that leads to production inefficiency. An accuracy of the demand forecast significantly affects safety stock and inventory levels, inventory holding costs, and customer service levels.

When the demand is highly seasonal, it is unlikely that an accurate forecast can be obtained without the use of an appropriate forecasting model. The demand forecast is one among several critical inputs of a production planning process. When the forecast is inaccurate, the obtained production plan will be unreliable, and may result in over- or understock problems. To avoid them, a suitable amount of safety stock must be provided, which requires additional investment in inventory and results in an increased inventory holding costs.

In order to solve the above-mentioned problems, systematic demand forecasting and production planning methods are proposed in this paper. A case study of a pressure container factory in Thailand is presented to demonstrate how the methods can be developed and implemented. This study illustrates that an improvement of demand forecasts and a reduction of total production costs can be achieved when the systematic demand forecasting and production planning methods are applied. The demand forecasting and production planning methods are proposed in the next section.

The background of the case study, including, products, production process, and the forecasting and production planning procedures being used in the factory, are briefly described in Section 3. The detailed analyses of the forecasting methods and the production planning method are explained in Section 4 and Section 5, respectively. Finally, the discussion and conclusion are presented in Section 6. 272 ScienceAsia 27 (2001) P ROPOSED D EMAND F ORECASTING PRODUCTION PLANNING METHODS AND The proposed demand forecasting and production planning methods are depicted in a step-by-step fashion in Fig. . Most factories produce a variety of products that can be categorized into product groups or families. Individual products in the same product group generally have some common characteristics. For example, they may have the same demand pattern and a relatively stable product mix. As a result, it is possible to forecast the aggregate demand of the product group first, and then disaggregate it into the demand of individual products. Since the forecast of the aggregate demand is more accurate than that of the individual demand1, it is initially determined in Step 1.

Then the demands of individual products are determined in Step 2 by multiplying the aggregate demand with the corresponding product mix that is normally known and quite constant. Since the demand forecasts are always subject to forecast errors, safety stocks are provided to avoid stock-out problems. Based on the standard deviation of the forecast errors and the required service level, the safety stocks for individual products are determined in Step 3. Production planning decisions are so complicated and important that they should not be subjectively and intuitively made.

Consequently, an appropriate production planning model should be formulated to determine the optimal decisions. With this model, its parameters, eg, demand forecasts, safety stocks, holding cost, overtime cost, machine capacity, inventory capacity, and available regular time and overtime, are entered or updated (Step 4). In step 5, the optimal decisions regarding the production quantities, inventory levels, and regular production time and overtime for each product in each production stage are obtained by solving the production planning model. Step 6 indicates that only the optimal production plan of the current month will be implemented.

After one month has elapsed, the demand forecasts and the production plan will be revised (by repeating Steps 1 to 5) according to a rolling horizon concept. BACKGROUND OF THE CASE STUDY The pressure container factory manufactures 15 products, ranging from 1. 25 to 50 kg of the capacity of pressurized gas. The products are divided into eight product groups, namely, Group 1 to Group 8. The first six groups have only two components, “head” and “bottom”, while the last two groups have three components, “head”, “bottom”, and “body”. The production process can be divided into five stages as shown in Fig. 2.

Stage 3 is only required to produce the products having three components (ie, those in Groups 7 and 8). Stage 4, the circumference welding, is found to be a bottleneck stage due to its long processing time. Presently monthly demand forecasts are subjectively determined by the Marketing Department based on past sales and expected future market conditions. No systematic method is used in forecasting. Using these forecasts and other constraints, such as availability of raw materials, equipment, and production capacity, the monthly production plan for a three-month period is intuitively determined without considering any cost factor.

This results in inaccurate demand forecasts and, subsequently, an inefficient production plan. Stage 1 Blanking 1) Forecast the monthly demands of each product group throughout the planning horizon of 12 months 2) Determine the demand for each individual product 3) Determine the safety stock for each individual product Stage 2 Forming of bottom and head Stage 3 Forming of body 4) Update the parameters in the production planning model Stage 4 Circumference welding 5) Run the planning model to obtain the optimal planning dicisions ) Roll the plan by repeating Steps 1 to 5 after one month has elapsed Stage 5 Finishing Fig 1. Proposed forecasting and planning steps. Fig 2. The production process to manufacture a pressure container. ScienceAsia 27 (2001) 273 FORECASTING METHODS Steps 1, 2, and 3 of the proposed forecasting and planning process are discussed in detail in this section. Firstly, the aggregate demand forecasts of eight product groups throughout the planning horizon of 12 months will be determined. Secondly, the demand forecasts of the product groups will be disaggregated into those of individual product.

Thirdly, the safety stocks of individual product will be calculated based on the forecast error. Aggregate Demand Forecasts of Product Groups The typical demand pattern of each product group is seasonal. As an example, Fig. 3 shows the demand pattern of Product Group 3. Thus, three forecasting models that are suitable for making seasonal demand forecasts are considered. They are Winter’s, decomposition and Auto-Regressive Integrated Moving Average (ARIMA) models. 2-5 Because of their simplicity, the Winter’s and decomposition models are initially used to forecast the aggregate demand of each product group.

If the Winter’s and decomposition models are inadequate (ie, the forecast errors are not random), the ARIMA model which is more complicated and perhaps more efficient will be applied. The Winter’s model has three smoothing parameters that significantly affect the accuracy of the forecasts. These parameters are varied at many levels using a computer program to determine a set of parameters that give the least forecast errors. There are two types of the decomposition model, namely, multiplicative and additive types. The former is selected since the demand pattern shows that the trend and seasonal components are dependent.

The forecast errors of the Winter’s and decomposition models are presented in Table 1. Based on the calculated mean square error (MSE) and the mean absolute percentage error (MAPE), it is seen that the decomposition model has lower Original Series (x 1000) 16 forecast errors in all product groups than the Winter’s model. Thus, it is reasonable to conclude that the decomposition model provides better demand forecasts than the other. One way to check whether the forecasting model is adequate is to evaluate the randomness of the forecast errors.

The auto-correlation coefficient functions (ACFs) of the errors from the decomposition model for several time lags at the significant level of 0. 05 of each product group are determined. The ACFs of Groups 1 and 3 are presented as examples in Fig. 4 and 5, respectively. The ACFs of Groups 4, 5, 6, 7, and 8 are similar to those of Group 1 in Table 1. Forecast errors of the Winter’s and decomposition models. MSE Products MAPE (%) Winter’s Decomposition Winter’s Decomposition 9,879,330 4,363,290 2,227,592 4,507,990 10,039,690 574,108 636,755 883,811 36. 14 48. 94 24. 25 30. 08 18. 80 53. 86 61. 99 46. 52 26. 97 31. 86 15. 97 23. 4 13. 14 34. 80 34. 45 28. 76 Group 1 16,855,149 Group 2 8,485,892 Group 3 5,433,666 Group 4 6,035,466 Group 5 23,030,657 Group 6 1,690,763 Group 7 2,034,917 Group 8 1,884,353 Estimated Autocorrelations 1 0. 5 coefficient 0 -0. 5 -1 0 4 8 lag 12 16 20 Fig 4. ACFs of the residuals from the decomposition model for Group 1. Estimated Autocorrelations 1 0. 5 16 demand 3 coefficient 0 8 -0. 5 4 -1 0 0 10 20 30 time index 40 50 60 0 4 8 lag 12 16 20 Fig 3. Actual demand of Group 3. Fig 5. ACFs of the residuals from the decomposition model for Group 3. 274 ScienceAsia 27 (2001) Fig 4, while those of Groups 2 and 3 are similar.

It can be seen from Fig. 4 that the ACFs of all lags are within the upper and lower limits, meaning that the errors are random. However, the ACF of lag 1 in Fig. 5 exceeds the upper limit. This indicates that auto-correlations do exist in the errors and that the errors are not random. From the ACFs, we can conclude that the decomposition model is adequate for forecasting the demands of Groups 1, 4, 5, 6, 7, and 8, but inadequate for forecasting those of Groups 2 and 3. Therefore, the ARIMA model is applied to Groups 2 and 3. From the original time series of the demand of Group 3 (in Fig. 3), and the ACFs of its original series (in Fig. ), it can be interpreted that the original series has a trend, and a high value of ACF of lag 12 indicates the existence of seasonality. 2 Hence, a non-seasonal first-difference to remove the trend and a seasonal first-difference to remove the strong seasonal spikes in the ACFs are tested. Fig. 7 shows the ACFs of the ARIMA (p,1,q)(P,1,Q) 12 model after applying the first difference. The nonseasonal plot indicates that there is an exponential decay and one significant ACF of lag 2. Thus, the AR(1) and MA(1) process denoted by ARIMA (1,1,1)(0,1,0)12 is identified. The ACFs of the residuals after applying this ARIMA model shown in Fig. reveals that there is a high value of ACF of lag 12. Therefore, the AR(1) and MA(1) process for the seasonal part or ARIMA (1,1,1)(1,1,1)12 can be identified. The ACFs of the residuals generated from this model are shown in Fig. 9. Since all ACFs are within the two significant limits, the ARIMA (1,1,1)(1,1,1)12 model is adequate. Using the Statgraphic program, the model coefficients can be determined. The demand forecast for Group 3 is presented in Eq. 1. Ft = 1. 197 X t ? 1 ? 0. 197 X t ? 2 + 0. 54408 X t ? 12 ? 0. 65126 X t ? 13 + 0. 10718 X t ? 14 + 0. 45592 X t ? 24 ? 0. 54574 X t ? 25 + 0. 08982 X t ? 26 ? 1. 6699et ? 1 ? 0. 7154et ? 12 + 0. 76332et ? 13 + 29. 34781 (1) where Ft is the demand forecast for period t Xt is the actual demand for period t et is the forecast error for period t Similarly, the forecasting model for Group 2 is ARIMA (3,0,0)(3,0,0). 12 The demand forecast of Group 2 is presented in Eq. 2. Estimated Autocorrelations for Original Series 1 Estimated Residual ACF 1 0. 5 0. 5 coefficient coefficient 0 0 -0. 5 -0. 5 -1 0 5 10 lag 15 20 25 -1 0 5 10 lag 15 20 25 Fig 6. ACFs of the actual demand for Group 3. Fig 8. ACFs of the residuals of ARIMA (1,1,1)(0,1,0)12 model for Group 3. Estimated Residual ACF 1

Estimated Autocorrelations for 1 Nonseasonal Differences 1 Seasonal Differences 1 0. 5 0. 5 coefficient coefficient 0 0 -0. 5 -0. 5 -1 0 5 10 lag 15 20 25 -1 0 5 10 lag 15 20 25 Fig 7. ACFs after first differencing for Group 3. Fig 9. ACFs of the residuals of ARIMA (1,1,1)(1,1,1)12 model for Group 3. ScienceAsia 27 (2001) 275 Ft = 0. 36951X t? 1 + 0. 30695X t? 2 – 0. 18213X t? 3 + 0. 20132 X t? 12 ? 0. 07439 X t? 13 ? 0. 06180 X 14 + 0. 03667 X t? 15 ? 0. 03325X t? 24 + 0. 01228 X t? 25 + 0. 01021X t? 26 ? 0. 00606 X t? 27 + 0. 68660 X t? 36 ? 0. 25371X t? 37 ? 0. 21075X t? 38 + 0. 12505X t? 39 + 354. 4515 2) The forecast errors of the decomposition and ARIMA models for Groups 2 and 3 are presented in Table 2. It reveals that the ARIMA model has lower Table 2. Forecast errors of the decomposition and ARIMA models. MSE Products Group 2 Group 3 Decomposition ARIMA 4,363,290 2,227,592 3,112,974 1,235,788 MAPE (%) Decomposition ARIMA 31. 86 15. 97 29. 05 13. 18 MSE and MAPE than the decomposition model. Therefore, the ARIMA model should be used to forecast the aggregate demands of Groups 2 and 3. For other product groups, however, the decomposition model should be used because it is more simple yet still adequate.

The comparison of the demand forecast errors obtained from the forecasting models and those from the current practice of the marketing department (as presented in Table 3) indicates that the errors of the forecasting models are substantially lower than those of the current practice. Demand Forecasts of Individual Products The demand forecast of product i for period t, dit, is obtained by multiplying the aggregate demand forecast of the product group (obtained from the previous steps) by the corresponding product mix (as presented in Table 4). Table 3. Forecast errors of the current practice, decomposition, and ARIMA models.

MSE Product Group 1 Group 2 Group 3 Group 4 Group 5 Group 6 Group 7 Group 8 Current practice Decomposition 16,672,342 4,394,693 4,988,962 4,754,572 19,787,102 795,621 849,420 1,060,301 9,879,330 4,507,990 10,039,690 574,108 636,755 883,811 ARIMA 3,112,974 1,235,788 MAPE (%) Current practice Decomposition 30. 58 34. 68 23. 50 25. 73 17. 54 42. 70 38. 36 37. 93 26. 97 23. 24 13. 14 34. 80 34. 45 28. 76 ARIMA 29. 05 13. 18 – Table 4. Product mix. Product group Product 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1 0. 17 0. 20 0. 26 0. 23 0. 14 1. 0 0. 53 0. 47 0. 65 0. 35 1. 0 1. 0 1. 0. 3 0. 7 2 3 4 5 6 7 8 276 ScienceAsia 27 (2001) Calculation of Safety Stock The safety stocks of finished products must be provided to protect against stock-out problems due to inaccurate demand forecasts. Based on the forecast errors obtained from the demand forecasting models, the amount of the safety stock is calculated using the following formula. 12 SSit = sf * ? j * ? ij (3) PRODUCTION PLANNING METHOD The production planning model is developed by initially defining decision variables and parameters, and then mathematically formulating the production planning model.

Step 4 of the method requires that the model parameters be estimated and entered into the model. The model is solved for the optimal solution (Step 5). Step 6 recommends that the model parameters are updated, and the model is solved again after one planning period has passed. The production planning problem of the factory under consideration belongs to the class of multistage, multi-item, capacitated production planning model. The models in this class have been discussed extensively in. 6-11 They differ in assumptions, objectives, constraints, and solution methods.

Our production planning model is a modification of the multi-stage, multi-product model discussed in Johnson and Montgomery. 6 Its objective is to minimize the total overtime and inventory holding costs. Costs of laying off and rehiring are not considered because laying off and rehiring are not allowed according to the labor union regulation. Since the production cost is time-invariant and all demands must be satisfied, the regular time production cost is thus not included in the objective function.

Relevant parameters and decision variables are defined as follows: Parameters : hik = Holding cost per unit of product i at stage k (baht/unit/period) co = Cost per man-hour of overtime labor (baht/man-hour) dit = Demand forecast of product i for period t (units) aik = Processing time for one unit of product i at stage k (hours/unit) (rm)kt = Total available regular time excluding preventive maintenance and festival days at stage k for period t (man-hours) (om)kt = Total available overtime excluding preventive maintenance and festival days at stage k for period t (man-hours) W = Warehouse capacity (units) SSit = Safety stock of product i for period t (units) Iik0 = Initial inventory of product i at stage k (units) N = Total number of products (15 products) T = Total number of periods in the planning horizon (12 periods) K = Total number of stages (5 stages) where SSit = Required safety stock level of product i for period t sf = Safety factor = 1. 64 for a required service level of 95 % of the standard normal distribution ? j = Standard deviation of forecast errors of Group j. ?ij = Product mix of Product i in Group j.

Since the errors of the recommended demand forecasting models are lower than those of the current practice, it is clear that SSit based on the use of the models must be lower than that determined from the current practice (assuming that the service levels of both cases are the same). Table 5 presents the required safety stocks of the current practice and the recommended forecasting models at 95 % service level. Table 5. Required safety stock of current practice and of recommended forecasting models. Safety stock (units) Product 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Current practice 1,138 1,339 1,741 1,540 937 3,438 1,941 1,722 2,324 1,252 7,295 1,463 1,511 507 1,182 Recommended forecasting models 887 1,043 1,356 1,200 730 2,905 979 868 2,274 1,224 5,258 1,245 1,323 460 1,072 ScienceAsia 27 (2001) 277

Decision variables: Xikt = Quantity of product i to be produced at stage k in period t (units) Iikt = Inventory of product i at stage k at the end of period t (units) Rkt = Regular time used at stage k during period t (man-hours) Okt = Overtime used at stage k during period t (man-hours) LP model: Minimize Z = ? ? ? hik Iikt + ? ? co Okt , (4) i =1 k =1 t =1 k =1 t =1 N K T K T Eq. 7 represents the material balance constraint in Stage 3, which produces the body of threecomponent products, for Products 13, 14, and 15. Constraint (13) must be included since the finished products are very bulky and require significant warehouse space that is quite limited.

Work-inprocess inventory does not require significant storage space because it can be stacked. The non-negativity constraint (16) ensures that shortages of work-inprocess inventory do not occur. Input Parameters The initial inventory of product i at stage k, Iik0, is collected from real data of work-in-process or finished good inventories on the factory floor at the beginning of the planning horizon. The inventory holding cost of product i at stage k, hik, is estimated by assuming that the annual inventory holding cost is 25% of the cost per unit of the product at the respective production stage. Since the cost per unit is constant over the planning horizon, the annual inventory holding cost is time-invariant.

The factory has enough space in the warehouse to store not more than 40,000 units of finished products. The total available regular time, (rm)kt, is estimated based on the fact that the factory is normally operated 16 hours a day and six days a week, and the total available overtime, (om)kt, is calculated by assuming that the overtime could not be more than six hours a day. The overtime cost, co, is assumed to be constant throughout the planning horizon, and is estimated to be 60 Baht per man-hour. After all related parameters have been estimated and entered into the planning model, the optimal values of all decision variables are calculated using the LINGO software. The computation time takes less than one minute on a Pentium PC.

Results of the Production Planning Models with Different Levels of Safety Stock In this section, two production planning models with different safety stock levels (as shown in Table 5) are solved to determine the total cost savings when the recommended forecasting models are applied in place of the current practice. The inventory holding, overtime, and total costs of both models are presented in Table 6. Based on the optimal total cost of the current practice (4,078,746 Baht per year) and the optimal total cost of the recommended forecasting models (3,541,772 Baht per year), the total cost saving is 536,974 Baht per year, or 13. 2 %. It can be also seen Subject to – Finished product requirement constraints I i 5,t? 1 + X i 5t ? I i 5t = dit – ? i, t ; k = 5, (5) Material balance between stages constraints ? i, t ; k = 4, (6) (7) ? i, t ; k = 2, (8) ? i, t ; k = 1, (9) I i 4 ,t? 1 + X i 4 t ? I i 4 t = X i 5t I i 3,t? 1 + X i 3t ?

I i 3t = X i 4 t ?t ; i = 13, 14, 15; k = 3, I i 2,t? 1 + X i 2t ? I i 2t = X i 4 t I i1,t? 1 + X i1t ? I i1t = X i 2t Capacity constraints ? aik X ikt ? Rkt + Okt i= 1 N ?k , t , (10) – Available regular and overtime constraints. Rkt ? (rm) kt Okt ? ( om) kt ?k , t , ? k , t , (11) (12) – Inventory capacity of finished product constraints. ? I ikt ? W i= 1 N ?t ; k = 5, (13) – Safety stock of finished product constraints. I ikt ? SS it ?i, t ; k = 5, (14) – Non-negativity conditions X ikt ? 0 I ikt ? 0 ?i, k , t , ? i, t ; k = 1, 2, 3, 4 (15) (16) 278 ScienceAsia 27 (2001) Table 6. Comparison of the optimal costs of production planning models.

Optimal costs (Baht/year) Model based on the current practice Inventory holding cost Overtime cost Total cost 2,117,051 1,961,695 4,078,746 Model based on recommended forecasting models 1,775,552 1,766,220 3,541,772 REFERENCES 1. Nahmias S (1993) Production and Operations Analysis, 2nd ed, Irwin, New York. 2. Vandaele W (1983) Applied Time Series and Box-Jenkins Models, Academic Press, New York. 3. Winters PR (1960) Forecasting Sales by Exponentially Weighted Moving Average. Management Science 6(4), 324-42. 4. Box GE and Jenkins GM (1970) Time Series Analysis, Forecasting, and Control, Holden-Day, San Francisco. 5. Makridakis S Wheelwright SC and McGee VE (1983) Forecasting Methods and Applications, 2nd ed, John Wiley & Sons, New York. 6.

Johnson LA and Montgomery DC (1974) Operations Research in Production Planning, Scheduling, and Inventory Control, John Wiley & Sons, New York. 7. Bullington P McClain J and Thomas J (1983) Mathematical Programming Approaches to Capacity Constrained MRP Systems: Review, Formulation, and Problem Reduction. Management Science 29(10). 8. Gabbay H (1979) Multi-Stage Production Planning. Management Science 25(11), 1138-48. 9. Zahorik A Thomas J and Trigeiro W (1984) Network Programming Models for Production Scheduling in MultiStage, Multi-Item Capacitated Systems. Management Science 30(3), 308-25. 10. Lanzanuer V (1970) Production and Employment Scheduling in Multi-Stage Production Systems. Naval Research Logistics Quarterly 17(2), 193-8. 11.

Schwarz LB (ed) (1981) Multi-level Production and Inventory Control Systems: Theory and Practice, North-Holland, New York. 12. Tersine RJ (1994) Principles of Inventory and Materials Management, 4th ed, Prentice Hall, New Jersey. that the optimal inventory holding cost and overtime cost in the production planning model based on the recommended forecasting models are almost equal which indicates that the model can efficiently achieve a tradeoff between both costs. Normally, the optimal decisions in the first planning period will be implemented. After the first period has passed, the new forecasts will be determined, and the model parameters will be updated.

The updated model is solved again to determine the optimal decisions in the current period. This is called a rolling horizon concept. However, the details and results of this step are not shown in this paper. DISCUSSION AND CONCLUSION The ARIMA model provides more reliable demand forecasts but it is more complicated to apply than the decomposition model. Therefore the ARIMA model should be used only when the decomposition model is inadequate. When compared against those of the current practice of the company, the errors of our selected models are considerably lower. This situation can lead to substantial reductions in safety stocks. Consequently, the lower safety stocks result in decreased inventory holding and overtime costs.

The results of the production planning model are of great value to the company since the model can determine the optimal overtime work, production quantities, and inventory levels that yield the optimal total overtime and holding costs. The production planning method is more suitable than the existing one that does not consider any cost factors. Moreover, it has been proven that an application of appropriate forecasting techniques can reduce total inventory holding and overtime costs significantly. In conclusion, this paper demonstrates that an improvement in demand forecasting and production planning can be achieved by replacing subjective and intuitive judgments by the systematic methods.