STARTUP Company Analysis

2Garments_worker-Copy1

Work in Progress…

Garments Production

g.jpg

Attribute Information:

  1. date : Date in MM-DD-YYYY
  2. day : Day of the Week
  3. quarter : A portion of the month. A month was divided into four quarters
  4. department : Associated department with the instance
  5. team : Associated team number with the instance
  6. no_of_workers : Number of workers in each team
  7. no_of_style_change : Number of changes in the style of a particular product
  8. targeted_productivity : Targeted productivity set by the Authority for each team for each day.
  9. smv : Standard Minute Value, it is the allocated time for a task
  10. wip : Work in progress. Includes the number of unfinished items for products
  11. overtime : Represents the amount of overtime by each team in minutes
  12. incentive : Represents the amount of financial incentive (in BDT) that enables or motivates a particular course of action.
  13. idletime : The amount of time when the production was interrupted due to several reasons
  14. idlemen : The number of workers who were idle due to production interruption
  15. actual_productivity : The actual % of productivity that was delivered by the workers. It ranges from 0-1.

Load Libraries and Dataset

In [2]:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import r2_score, mean_absolute_error, mean_squared_error
from scipy.stats import pearsonr
In [13]:
df = pd.read_csv('garments_worker_productivity.csv')
print(df.shape)
df.head()
(1197, 15)
Out[13]:
date quarter department day team targeted_productivity smv wip over_time incentive idle_time idle_men no_of_style_change no_of_workers actual_productivity
0 1/1/2015 Quarter1 sweing Thursday 8 0.80 26.16 1108.0 7080 98 0.0 0 0 59.0 0.940725
1 1/1/2015 Quarter1 finishing Thursday 1 0.75 3.94 NaN 960 0 0.0 0 0 8.0 0.886500
2 1/1/2015 Quarter1 sweing Thursday 11 0.80 11.41 968.0 3660 50 0.0 0 0 30.5 0.800570
3 1/1/2015 Quarter1 sweing Thursday 12 0.80 11.41 968.0 3660 50 0.0 0 0 30.5 0.800570
4 1/1/2015 Quarter1 sweing Thursday 6 0.80 25.90 1170.0 1920 50 0.0 0 0 56.0 0.800382

EDA

Inspect, Clean, and Validate. Show underlying patterns and relationships within datasets.

Firstly we need to show all the variable distribution for reference.

In [94]:
# we will not include date in this visualization
df.drop(columns=['date']).hist(figsize=(10, 7))
plt.tight_layout()
plt.show()

Check for skewness

Skewness is a measure of asymmetry of a distribution. Important Notes:

  • between -0.5 and 0.5, fairly symmetrical
  • between -1 and -0.5 or between 0.5 and 1, moderately skewed
  • less than -1 or greater than 1, highly skewed
In [95]:
# selected columns only
df_int = ['team', 'targeted_productivity', 'smv', 'wip', 'over_time', 'incentive', 'no_of_workers', 'actual_productivity']
for i in df_int:
    print(i + ' ==> ' + str(round(df[i].skew(),2)))
team ==> 0.01
targeted_productivity ==> -2.14
smv ==> 0.41
wip ==> 10.85
over_time ==> 0.67
incentive ==> 15.79
no_of_workers ==> -0.11
actual_productivity ==> -0.81

Kurtosis()

kurtosis determines the heaviness of the distribution tails. Determine the volume of the outlier.

  • If the distribution is tall and thin it is called a leptokurtic distribution(Kurtosis > 3). Values in a leptokurtic distribution are near the mean or at the extremes.
  • A flat distribution where the values are moderately spread out is called platykurtic(Kurtosis <3) distribution.
  • A distribution whose shape is in between a leptokurtic distribution and a platykurtic distribution is called a mesokurtic(Kurtosis=3) distribution. A mesokurtic distribution looks more close to a normal distribution.

‘Note’

  1. High kurtosis in a df set is an indicator that df has heavy outliers.
  2. Low kurtosis in a df set is an indicator that df has lack of outliers.
  3. If kurtosis value + means pointy and — means flat.
In [97]:
df.kurt()
C:\Users\Toto\AppData\Local\Temp\ipykernel_27732\1257127604.py:1: FutureWarning:

Dropping of nuisance columns in DataFrame reductions (with 'numeric_only=None') is deprecated; in a future version this will raise TypeError.  Select only valid columns before calling the reduction.

Out[97]:
team                      -1.223906
targeted_productivity      5.613701
smv                       -0.795346
wip                      141.332609
over_time                  0.424364
incentive                299.032462
idle_time                442.638160
idle_men                 102.962869
no_of_style_change         8.181490
no_of_workers             -1.788108
actual_productivity        0.333227
dept_mapped               -1.905055
month                     -1.057430
dtype: float64
  1. teams Activity seems fairly distributed and flat
  2. target productivity Heavily left skewed which is a great. Target productivy should be in increasing value.
  3. smv Distribution is faily symmetrical
  4. wip Heavily right skewed, as seen in above graph outliers are affecting the distribution!. Kurtosis at 110 indicating that the distribution is very tall. Also distribution has heavy outliers.
  5. over_time Moderately skewed
  6. incentive Heavily right skewed. There must some extremely high incentives!. Kurtosis at 299 we should do an investigation with the outliers.
  7. no_of_worker Looks like a bimodal distribution, we can form a clusters/groups for this kind of distribution.
  8. Actual_productiviy Moderately left skewed. This is a good sign.

Dataset general information

In [14]:
df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1197 entries, 0 to 1196
Data columns (total 15 columns):
 #   Column                 Non-Null Count  Dtype  
---  ------                 --------------  -----  
 0   date                   1197 non-null   object 
 1   quarter                1197 non-null   object 
 2   department             1197 non-null   object 
 3   day                    1197 non-null   object 
 4   team                   1197 non-null   int64  
 5   targeted_productivity  1197 non-null   float64
 6   smv                    1197 non-null   float64
 7   wip                    691 non-null    float64
 8   over_time              1197 non-null   int64  
 9   incentive              1197 non-null   int64  
 10  idle_time              1197 non-null   float64
 11  idle_men               1197 non-null   int64  
 12  no_of_style_change     1197 non-null   int64  
 13  no_of_workers          1197 non-null   float64
 14  actual_productivity    1197 non-null   float64
dtypes: float64(6), int64(5), object(4)
memory usage: 140.4+ KB

task:

Work in Progress…

Rename department column wrong typo’s and convert to categorical dtypes

In [15]:
df.department.value_counts()
Out[15]:
sweing        691
finishing     257
finishing     249
Name: department, dtype: int64
In [16]:
df['department'] = df['department'].replace(['sweing'], 'sewing')
df['department'] = df['department'].replace(['finishing '], 'finishing')
df['department'] = df.department.astype('category')

df.department.value_counts()
Out[16]:
sewing       691
finishing    506
Name: department, dtype: int64

Duplicates

In [17]:
# variable that store duplicates exactly the same as another row
dups = df.duplicated()
# count duplicates
dups.value_counts()
Out[17]:
False    1197
dtype: int64

Theres no duplicates.

Clean the Data (Null Values)

Check for the frequency of nulls in all the columns

In [18]:
df.isna().sum()
Out[18]:
date                       0
quarter                    0
department                 0
day                        0
team                       0
targeted_productivity      0
smv                        0
wip                      506
over_time                  0
incentive                  0
idle_time                  0
idle_men                   0
no_of_style_change         0
no_of_workers              0
actual_productivity        0
dtype: int64

Theres a large proportion of null values in wip(42%). The first approach is to investigate how this data was collected or structured.

We need to undestand what type of missing data we are dealing:

  • Structurally missing data
  • Missing Completely at Random (MCAR)
  • Missing at Random (MAR)
  • Missing Not at Random (MNAR)

Check the shape of finishing and sewing department seperately.

In [19]:
finishing = df[df['department']=='finishing']
sewing = df[df['department']=='sewing']

print(finishing.shape)
sewing.shape
(506, 15)
Out[19]:
(691, 15)

Check the total number of nulls in wip.

In [20]:
finishing.wip.isna().sum()
Out[20]:
506

Nova! all of the NaN entries are comming from finishing department. What we need to so now is to ask and verify if there is really no pending works(wip) in the finishing department, just to be sure. Since there is no one to ask for this project I will conclude that this missing data is labeled as Structurally Missing thus there is no wip in the finishing department.

The graph shows that there is no records of

Now were gonna impute these NaN values. We have a lot of techniques in imputing Nan’s (ex. time series LOCF and NOCB, pairwise, listwise etc.) in this scenario we will convert these NaN in to zero.

In [21]:
# Converting wip NaN values to zero
df.wip.fillna(0, inplace=True)

Visualize the null datapoints

We convert first all nulls into 0 before plotting thats because matplot library does’nt read nulls so if we plot our variables without converting, null will not show in the plot.

In [26]:
# Code preparation for plotting.
# Mapping departent to 0 and 1 as integer 
df['dept_mapped'] = df.department.map({
                                'finishing':0,
                                'sewing':1})
# convert to int dtype
df.dept_mapped  = df.dept_mapped.astype('int64')
In [27]:
# plt.figure(figsize=(1, 1)) 
g = sns.lmplot(x='dept_mapped', y='wip', 
               data=df, 
               x_jitter=0.15, 
               y_jitter=0.15, 
               fit_reg=False, 
               scatter_kws={'alpha':0.2})

g.set(xticks=range(2))
g.set_xticklabels(['finishing', 'sewing'])
plt.show()

Convert date to datetime dtype

In [28]:
# date format is MM-DD-YYYY
df['date'] = pd.to_datetime(df.date, format="%m/%d/%Y")
  • Augment data with additional columns
In [29]:
# add month
df['month'] = df['date'].dt.month
In [30]:
# Adding `work_week` columns. It's easier to analyze production data per work week.
# create a list of our conditions
conditions = [
    
    #January
    (df['month'] == 1) & (df['quarter'] == 'Quarter1'), #ww1
    (df['month'] == 1) & (df['quarter'] == 'Quarter2'), #ww2
    (df['month'] == 1) & (df['quarter'] == 'Quarter3'), #ww3
    (df['month'] == 1) & (df['quarter'] == 'Quarter4'), #ww4
    (df['month'] == 1) & (df['quarter'] == 'Quarter5'), #ww4
    
    #February
    (df['month'] == 2) & (df['quarter'] == 'Quarter1'), #ww5
    (df['month'] == 2) & (df['quarter'] == 'Quarter2'), #ww6
    (df['month'] == 2) & (df['quarter'] == 'Quarter3'), #ww7
    (df['month'] == 2) & (df['quarter'] == 'Quarter4'), #ww8
    (df['month'] == 2) & (df['quarter'] == 'Quarter5'), #ww8
    
    #March
    (df['month'] == 3) & (df['quarter'] == 'Quarter1'), #ww9
    (df['month'] == 3) & (df['quarter'] == 'Quarter2'), #ww_10
    (df['month'] == 3) & (df['quarter'] == 'Quarter3'), #ww_11
    (df['month'] == 3) & (df['quarter'] == 'Quarter4'), #ww_12
    (df['month'] == 3) & (df['quarter'] == 'Quarter5'), #ww_12
    ]

values = ['ww1','ww2','ww3','ww4','ww4',
          'ww5','ww6','ww7','ww8','ww8',
          'ww9','ww_10','ww_11','ww_12','ww_12']

df['work_week'] = np.select(conditions, values)

Quater column to categorical dtypes

In [31]:
quarter_order = ['Quarter1', 'Quarter2', 'Quarter3', 'Quarter4', 'Quarter5' ]
df.quarter = pd.Categorical(df.quarter, quarter_order, ordered=True)
# verify quarter order
df.quarter.unique()
Out[31]:
['Quarter1', 'Quarter2', 'Quarter3', 'Quarter4', 'Quarter5']
Categories (5, object): ['Quarter1' < 'Quarter2' < 'Quarter3' < 'Quarter4' < 'Quarter5']

Day column to categorical dtypes

In [32]:
df.day.unique()
Out[32]:
array(['Thursday', 'Saturday', 'Sunday', 'Monday', 'Tuesday', 'Wednesday'],
      dtype=object)
In [33]:
day_order = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Saturday', 'Sunday' ]
df.day = pd.Categorical(df.day, day_order, ordered=True)
# verify day order
df.day.unique()
Out[33]:
['Thursday', 'Saturday', 'Sunday', 'Monday', 'Tuesday', 'Wednesday']
Categories (6, object): ['Monday' < 'Tuesday' < 'Wednesday' < 'Thursday' < 'Saturday' < 'Sunday']

Inspect zero value (60% and up)

Let’s investigate this zero values.

In [37]:
# how many zeros are there in each columns?
# (df == 0).sum(axis=0) # uncomment to show result

print('Proportion of zeros: ')
(df == 0).sum(axis=0)  / len(df) * 100
Proportion of zeros: 
Out[37]:
date                      0.000000
quarter                   0.000000
department                0.000000
day                       0.000000
team                      0.000000
targeted_productivity     0.000000
smv                       0.000000
wip                      42.272348
over_time                 2.589808
incentive                50.459482
idle_time                98.496241
idle_men                 98.496241
no_of_style_change       87.719298
no_of_workers             0.000000
actual_productivity       0.000000
dept_mapped              42.272348
month                     0.000000
work_week                 0.000000
dtype: float64

We have a very large proportion of zero value in wip, incentive, idle_time, idle_men, and no_of_style_change.

  • wip This 42% of zeros are came from finishing department alternately the remaining 58% are work in progress by the sewing department.
  • incentive 50% or half of the total production days got incentive
  • idle_time This large proportion is a good indication that the the production is running smoothly.
  • idle_men Same as idle_time, most of the teams/workers are working deligently. 98% are zero value.
  • no_of_style_change 87.72% are zero values. Seems fair there is only 12% of change of product style that caused idle_time. Changing style might result to idle_time if not properly executed.
1. Does no_of_style_change produce idle_time?

Frequency of unique value in no_of_style_change

In [69]:
df.no_of_style_change.value_counts()
Out[69]:
0    1050
1     114
2      33
Name: no_of_style_change, dtype: int64

Proportion

In [66]:
import plotly.express as px
fig = px.pie(df, 'no_of_style_change')

fig.update_layout(title="Proportion of no_of_style_change")
fig.update_traces(textposition='inside',
                  textinfo='percent+label', showlegend=True)

fig.update_traces(pull= 0.02)
fig.show('png')

Correlation between no_of_style_change and idle_time.

In [70]:
from scipy.stats import pearsonr
corr, p = pearsonr(df.no_of_style_change, df.idle_time)
corr, p
Out[70]:
(-0.011597866663013127, 0.6885278961213044)

There’s no significant correlation between no_of_style_change and idle_time. Also the pval is at very high value.

In [72]:
# visualize
# plt.figure(figsize=(1, 1)) 
g = sns.lmplot(x='no_of_style_change', y='idle_time', 
               data=df, 
               x_jitter=0.15, 
               y_jitter=0.15, 
               fit_reg=False, 
               scatter_kws={'alpha':0.5})

g.set(xticks=range(3))
g.set_xticklabels(df.no_of_style_change.unique())
g.set(ylim=(0,10))
plt.show()
  • There is 147(12.28%) total number change in production
  • Changing style can cause idle time up to 8 minutes

2. Targeted productivity vs Actual productivity

Note Prioritizing the actual productivity, we group our dataset by work_week then we get the mean instead of median(we use median for heavily skew distribution). Checking the shape of its overall distribution(ww1-ww10) we can see a moderately skew shape(skewnew at -.81 kurtusis at .33). To be more precise we can check the distribution per work_week but I think it’s not necessary here.

In [73]:
target_actual = df.groupby('work_week').mean()
target_actual[['targeted_productivity', 'actual_productivity']]
Out[73]:
targeted_productivity actual_productivity
work_week
ww1 0.774603 0.779225
ww2 0.746617 0.749937
ww3 0.715625 0.717431
ww4 0.704678 0.767010
ww5 0.737345 0.770341
ww6 0.723874 0.741566
ww7 0.727551 0.690275
ww8 0.729752 0.669766
ww9 0.718595 0.705212
ww_10 0.720879 0.737224

Visualization

In [74]:
# setting variables
# target productivity average per workweek
tp_avg = df.groupby('work_week')['targeted_productivity'].mean()
# actual productivity average per workweek
ap_avg = df.groupby('work_week')['actual_productivity'].mean()
In [82]:
plt.figure(figsize=(10,5))

plt.plot(tp_avg.index, tp_avg, marker='o')
plt.plot(ap_avg.index, ap_avg, marker='o')

plt.xlabel('work weeks')
plt.ylabel('productivity score')
plt.legend(['targeted productivity', 'actual productivity'])

plt.show()

We have a very good ww4 and ww5 however after ww5 our productivity crashed to a very low level.

In [85]:
# Checking the reason behind the crash
idle = df.groupby('work_week').sum()
idle['idle_time']
Out[85]:
work_week
ww1        0.0
ww2        0.0
ww3        0.0
ww4        0.0
ww5      810.0
ww6        0.0
ww7       38.0
ww8        8.0
ww9       18.0
ww_10      0.0
Name: idle_time, dtype: float64

We have a major downtime during ww5 that caused the fall of actual productivity.

4. How much over_time does production have per work_week? Also the number of workers?
In [86]:
# set variables
ww = df.groupby('work_week').sum() # we use groupby then sum
ww['cummulative'] = ww.over_time.cumsum()
ww[['over_time', 'cummulative', 'no_of_workers']]
Out[86]:
over_time cummulative no_of_workers
work_week
ww1 642450 642450 4304.5
ww2 777630 1420080 4303.5
ww3 722160 2142240 4216.0
ww4 895050 3037290 5720.5
ww5 479160 3516450 4182.0
ww6 370620 3887070 4155.0
ww7 306000 4193070 3436.5
ww8 471980 4665050 4210.0
ww9 491520 5156570 4057.0
ww_10 310680 5467250 2843.0
  • There are 642450 overtime minutes from work week1 with 4305 workers. An average of 2.48 OT hours per worker in ww1. (642450 / 4305 we will get 149 minutes then we divide by 60 to get 2.48 hours)
Visualization

Over time per work week and it’s cummulative value.

In [87]:
#
plt.figure(figsize=(14, 6)) 

# set variables
x = ww.index
y = ww.over_time

# plot1
# sum of overtime per work_week
ax1 = plt.subplot(1,2,1)
sns.barplot(x = x, y = y, data=df)
ax1.set_xticklabels(ax1.get_xticklabels(),rotation = 80)
plt.title('Overtime per Work Week ')

for i in ax1.containers:
    ax1.bar_label(i,)


#plot2
# cumulative sum of work_week
ax2 = plt.subplot(1,2,2)
sns.lineplot(x=x, y=ww.over_time.cumsum(), marker='o', )
plt.title('Cumulative Overtime')

plt.tight_layout()
plt.show()
5. What the best team so far?

note: Productivity is scaled from 0-1(0-100%)

  • We will group by team then find the mean(avg) of it’s productivity.

Note we get the mean instead of median. I’m prioritizing the actual productivity. Checkingthe shape of its distribution we can see a moderately skew(skewnew at -.81 kurtusis at .33). Unless it’s heavily skew then we can use the median.

In [88]:
team = df.groupby('team').mean() # groupby then get the mean
team['diff_in_productivity'] = team.actual_productivity - team.targeted_productivity

# limit to 2 decimal places
team = team.round(2)

team[['targeted_productivity', 'actual_productivity', 'diff_in_productivity']]
Out[88]:
targeted_productivity actual_productivity diff_in_productivity
team
1 0.75 0.82 0.07
2 0.74 0.77 0.03
3 0.74 0.80 0.06
4 0.72 0.77 0.05
5 0.67 0.70 0.02
6 0.73 0.69 -0.05
7 0.71 0.67 -0.05
8 0.71 0.67 -0.03
9 0.76 0.73 -0.02
10 0.74 0.72 -0.02
11 0.70 0.68 -0.02
12 0.77 0.78 0.00

Top 3 Productive Team

In [89]:
team.actual_productivity.sort_values(ascending=False).head(3)
Out[89]:
team
1     0.82
3     0.80
12    0.78
Name: actual_productivity, dtype: float64
Visualization
In [90]:
#
x = team.index
y = team.actual_productivity

plt.figure(figsize=(8,5))
ax1 = plt.subplot()
sns.barplot(x=x, y=y, palette = 'deep')
plt.title('Productivity by Team')

for i in ax1.containers:
    ax1.bar_label(i,)

plt.tight_layout()    
plt.show()

Team productivity by department

In [91]:
team_dept = df.groupby(['team','department']).mean() 
team_dept['diff_productivity'] = team_dept.actual_productivity - team_dept.targeted_productivity
team_dept = team_dept.round(2)

# Show first 3 teams
team_dept[['targeted_productivity', 'actual_productivity', 'diff_productivity']].head(6)
Out[91]:
targeted_productivity actual_productivity diff_productivity
team department
1 finishing 0.75 0.83 0.08
sewing 0.74 0.82 0.07
2 finishing 0.75 0.78 0.03
sewing 0.73 0.76 0.03
3 finishing 0.74 0.85 0.11
sewing 0.74 0.78 0.03

Visualization

In [92]:
# code preparation for visualization
team_act_prod = []
for i in team_dept.actual_productivity:
    team_act_prod.append(i)

sewing = team_act_prod[1::2]
finishing= team_act_prod[0::2]

# team_dept is a 2d list
# example code for accessing 2d indeces
# team 2 finishing dept would be:
# team_dept.actual_productivity[2][0]
In [93]:
#
fig, ax = plt.subplots(figsize=(12,6))

x = np.arange(1,13)
width = 0.35  # the width of the bars

rects1 = ax.bar(x - width/2, finishing, width, label= 'finishing')
rects2 = ax.bar(x + width/2, sewing, width, label='sewing')
ax.set_xticks(x)
ax.set_xlabel('Teams')
ax.set_ylabel('Actual Productivity')

def autolabel(rects):
    """Attach a text label above each bar in *rects*, displaying its height."""
    for rect in rects:
        height = rect.get_height()
        ax.annotate('{}'.format(height),
                    xy=(rect.get_x() + rect.get_width() / 2, height),
                    xytext=(0, 3),  # 3 points vertical offset
                    textcoords="offset points",
                    ha='center', va='bottom')

autolabel(rects1)
autolabel(rects2)
plt.tight_layout()
plt.legend(loc='upper center')
plt.show()

Statistical Analysis

Descriptive

Univiriate Analysis

1. What is the average and median overtime per work_week? .

In [111]:
ot_avg_ww = df.groupby('work_week').mean()['over_time']
ot_mdn_ww = df.groupby('work_week').median()['over_time']
ot_avg_ww, ot_mdn_ww
Out[111]:
(work_week
 ww1      5098.809524
 ww2      5846.842105
 ww3      6447.857143
 ww4      5234.210526
 ww5      4240.353982
 ww6      3338.918919
 ww7      3122.448980
 ww8      3900.661157
 ww9      4062.148760
 ww_10    3414.065934
 Name: over_time, dtype: float64,
 work_week
 ww1      6150.0
 ww2      5490.0
 ww3      6465.0
 ww4      5760.0
 ww5      4080.0
 ww6      2400.0
 ww7      2220.0
 ww8      3780.0
 ww9      3780.0
 ww_10    3360.0
 Name: over_time, dtype: float64)

2. Highest and lowest incentive? (except zero for lowest)

In [114]:
# highest incentive
inc_max = np.max(df.incentive)
inc_min = df.incentive[df.incentive !=0].min()
inc_max, inc_min
Out[114]:
(3600, 21)
In [ ]:
 
In [ ]:
 
In [116]:
df.month.unique()
Out[116]:
array([1, 2, 3], dtype=int64)
In [118]:
sns.boxplot(x='month', y='actual_productivity', data=df)
plt.show()
In [127]:
jan = df[df['month']==1]
feb = df[df['month']==2]
mar = df[df['month']==3]

plt.hist(jan.actual_productivity, alpha=.5, label='January')
plt.hist(feb.actual_productivity, alpha=.5, label='February')
plt.hist(mar.actual_productivity, alpha=.5, label='March')

plt.xlabel('Actual Productivity')
plt.ylabel('Count')
plt.legend()
plt.show()