Models updated

.gitignore

@@ -2,6 +2,8 @@
 data
 data_svm
 saved_models
+threshold_map
+models_info
 
 .python-version
 __pycache__

README.md

@@ -8,15 +8,126 @@ pip install -r requirements.txt
 
 ## How to use
 
-Generate data .csv files (run only once time or clean data folder before) :
+### Multiple folders and scripts are availables :
 
+
+- **fichiersSVD/\*** : all scene files information (zones of each scene, SVD descriptor files information and so on...).
+- **fichiersSVD_light/\*** : all scene files information (zones of each scene, SVD descriptor files information and so on...) but here with reduction of information for few scenes. Information used in our case.
+- **models/*.py** : all models developed to predict noise in image.
+- **data_svm/\*** : folder which will contain all *.train* & *.test* files in order to train model.
+- **saved_models/*.joblib** : all scikit learn models saved.
+- **models_info/*.md** : all markdown files generated to get quick information about model performance and prediction.
+
+### Scripts for generating data files
+
+Two scripts can be used for generating data in order to fit model :
+- **generate_data_svm.py** : zones are specified and stayed fixed for each scene
+- **generate_data_svm_random.py** : zones are chosen randomly (just a number of zone is specified)
+
+
+**Remark** : Note here that all python script have *--help* command.
+
+```
+python generate_data_svm.py --help
+
+python generate_data_svm.py --output xxxx --interval 0,20  --kind svdne --scenes "A, B, D" --zones "0, 1, 2" --percent 0.7 --sep : --rowindex 1
+```
+
+Parameters explained : 
+- **output** : filename of data (which will be split into two parts, *.train* and *.test* relative to your choices).
+- **interval** : the interval of data you want to use from SVD vector.
+- **kind** : kind of data ['svd', 'sdvn', 'sdvne']; not normalize, normalize vector only and normalize together.
+- **scenes** : scenes choice for training dataset.
+- **zones** : zones to take for training dataset.
+- **percent** : percent of data amount of zone to take (choose randomly) of zone
+- **sep** : output csv file seperator used
+- **rowindex** : if 1 then row will be like that 1:xxxxx, 2:xxxxxx, ..., n:xxxxxx
+
+### Train model
+
+This is an example of how to train a model
+
+```python
+python models/xxxxx.py --data 'data_svm/xxxxx.train' --output 'model_file_to_save'
+```
+
+### Predict image using model
+
+Now we have a model trained, we can use it with an image as input :
+
+```python
+python predict_noisy_image_svd_lab.py --image path/to/image.png --interval "x,x" --model saved_models/xxxxxx.joblib --mode 'svdn'
+```
+
+The model will return only 0 or 1 :
+- 1 means noisy image is detected.
+- 0 means image seem to be not noisy.
+
+### Predict scene using model
+
+Now we have a model trained, we can use it with an image as input :
+
+```python
+python prediction_scene.py --data path/to/xxxx.csv --model saved_model/xxxx.joblib --output xxxxx --scene xxxx
+```
+**Remark** : *scene* parameter expected need to be the correct name of the Scene.
+
+## Others scripts
+
+### Test model on all scene data
+
+In order to see if a model well generalized, a bash script is available :
+
+```bash
+bash testModelByScene.sh '100' '110' 'saved_models/xxxx.joblib' 'svdne'
 ```
-python generate_data.py
+
+Parameters list :
+- 1 : Begin of interval of data from SVD to use
+- 2 : End of interval of data from SVD to use
+- 3 : Model we want to test
+- 4 : Kind of data input used by trained model
+
+
+### Get treshold map 
+
+Main objective of this project is to predict as well as a human the noise perception on a photo realistic image. Human threshold is available from training data. So a script was developed to give the predicted treshold from model and compare predicted treshold from the expected one.
+
+```python
+python predict_noisy_image.py --interval "x,x" --model 'saved_models/xxxx.joblib' --mode ["svd", "svdn", "svdne"] --limit_detection xx
 ```
 
-## Models
+Parameters list :
+- **model** : mode file saved to use
+- **interval** : the interval of data you want to use from SVD vector.
+- **mode** : kind of data ['svd', 'sdvn', 'sdvne']; not normalize, normalize vector only and normalize together.
+- **limit_detection** : number of not noisy images found to stop and return threshold (integer).
+
+### Display model performance information
+
+Another script was developed to display into Mardown format the performance of a model.
+
+The content will be divised into two parts :
+- Predicted performance on all scenes
+- Treshold maps obtained from model on each scenes
+
+The previous script need to already have ran to obtain and display treshold maps on this markdown file.
+
+```python
+python save_model_result_in_md.py --interval "xx,xx" --model saved_models/xxxx.joblib --mode ["svd", "svdn", "svdne"]
+```
+
+Parameters list :
+- **model** : mode file saved to use
+- **interval** : the interval of data you want to use from SVD vector.
+- **mode** : kind of data ['svd', 'sdvn', 'sdvne']; not normalize, normalize vector only and normalize together.
+
+
+Markdown file is saved using model name into **models_info** folder.
+
+### Others...
 
-*Documentation will be implemented later...*
+All others bash scripts are used to combine and run multiple model combinations...
 
 ## How to contribute
 

ensemble_model_train.py

@@ -14,9 +14,14 @@ from sklearn.metrics import accuracy_score
 
 import sys, os, getopt
 
-output_model_folder = './saved_models/'
+current_dirpath = os.getcwd()
+output_model_folder = os.path.join(current_dirpath, 'saved_models')
 
 def get_best_model(X_train, y_train):
+    Cs = [0.001, 0.01, 0.1, 1, 10]
+    gammas = [0.001, 0.01, 0.1, 1]
+    param_grid = {'kernel':['rbf'], 'C': Cs, 'gamma' : gammas}
+    
     parameters = {'kernel':['rbf'], 'C': np.arange(1, 20)}
     svc = svm.SVC(gamma="scale", probability=True)
     clf = GridSearchCV(svc, parameters, cv=5, scoring='accuracy', verbose=10)
@@ -65,7 +70,7 @@ def main():
     not_noisy_df = dataset[dataset.ix[:, 0] == 0]
     nb_noisy = len(noisy_df.index)
     
-    final_df = pd.concat([not_noisy_df[0:nb_noisy], noisy_df[:]])
+    final_df = pd.concat([not_noisy_df[0:nb_noisy], noisy_df])
     #final_df = pd.concat([not_noisy_df, noisy_df])
     
     # shuffle data another time
@@ -95,7 +100,7 @@ def main():
     y_pred = ensemble_model.predict(X_test)
     print("**Test :** " + str(accuracy_score(y_test, y_pred)))
 
-    joblib.dump(ensemble_model, output_model_folder + p_output + '.joblib') 
+    joblib.dump(ensemble_model, output_model_folder + '/' + p_output + '.joblib') 
 
 if __name__== "__main__":
     main()

models/ensemble_model_train_v2.py

@@ -0,0 +1,113 @@
+from sklearn.model_selection import train_test_split
+from sklearn.model_selection import GridSearchCV
+from sklearn.linear_model import LogisticRegression
+from sklearn.ensemble import RandomForestClassifier, VotingClassifier
+from sklearn.neighbors import KNeighborsClassifier
+from sklearn.ensemble import GradientBoostingClassifier
+
+import sklearn.svm as svm
+from sklearn.utils import shuffle
+from sklearn.externals import joblib
+
+import numpy as np
+
+import pandas as pd
+from sklearn.metrics import accuracy_score
+
+import sys, os, getopt
+
+current_dirpath = os.getcwd()
+output_model_folder = os.path.join(current_dirpath, 'saved_models')
+
+def get_best_model(X_train, y_train):
+    Cs = [0.001, 0.01, 0.1, 1, 10, 20, 30]
+    gammas = [0.001, 0.01, 0.1, 1, 5, 10]
+    param_grid = {'kernel':['rbf'], 'C': Cs, 'gamma' : gammas}
+    
+    parameters = {'kernel':['rbf'], 'C': np.arange(1, 20)}
+    svc = svm.SVC(gamma="scale", probability=True, max_iter=10000)
+    clf = GridSearchCV(svc, parameters, cv=5, scoring='accuracy', verbose=10)
+
+    clf.fit(X_train, y_train)
+
+    model = clf.best_estimator_
+
+    return model
+
+
+def main():
+
+    if len(sys.argv) <= 1:
+        print('Run with default parameters...')
+        print('python smv_model_train.py --data xxxx --output xxxx')
+        sys.exit(2)
+    try:
+        opts, args = getopt.getopt(sys.argv[1:], "hd:o", ["help=", "data=", "output="])
+    except getopt.GetoptError:
+        # print help information and exit:
+        print('python smv_model_train.py --data xxxx --output xxxx')
+        sys.exit(2)
+    for o, a in opts:
+        if o == "-h":
+            print('python smv_model_train.py --data xxxx --output xxxx')
+            sys.exit()
+        elif o in ("-d", "--data"):
+            p_data_file = a
+        elif o in ("-o", "--output"):
+            p_output = a
+        else:
+            assert False, "unhandled option"
+
+    if not os.path.exists(output_model_folder):
+        os.makedirs(output_model_folder)
+
+    # get and split data
+    dataset = pd.read_csv(p_data_file, header=None, sep=";")
+
+     # default first shuffle of data
+    dataset = shuffle(dataset)
+    
+    # get dataset with equal number of classes occurences
+    noisy_df = dataset[dataset.ix[:, 0] == 1]
+    not_noisy_df = dataset[dataset.ix[:, 0] == 0]
+    nb_noisy = len(noisy_df.index)
+    
+    final_df = pd.concat([not_noisy_df[0:nb_noisy], noisy_df[:]])
+    #final_df = pd.concat([not_noisy_df, noisy_df])
+    
+    # shuffle data another time
+    final_df = shuffle(final_df)
+    
+    print(len(final_df.index))
+
+    y_dataset = final_df.ix[:,0]
+    x_dataset = final_df.ix[:,1:]
+
+    X_train, X_test, y_train, y_test = train_test_split(x_dataset, y_dataset, test_size=0.5, random_state=42)
+
+    svm_model = get_best_model(X_train, y_train)
+    knc_model = KNeighborsClassifier(n_neighbors=2)
+    gbc_model = GradientBoostingClassifier(n_estimators=100, learning_rate=1.0, max_depth=1, random_state=0)
+    lr_model = LogisticRegression(solver='liblinear', multi_class='ovr', random_state=1)
+    rf_model = RandomForestClassifier(n_estimators=100, random_state=1)
+
+    ensemble_model = VotingClassifier(estimators=[
+       ('lr', lr_model), 
+       ('knc', knc_model),
+       ('gbc', gbc_model),
+       ('svm', svm_model), 
+       ('rf', rf_model)], 
+       voting='soft', weights=[1, 1, 1, 1, 1])
+
+    ensemble_model.fit(X_train, y_train)
+
+    y_train_model = ensemble_model.predict(X_train)
+    print("**Train :** " + str(accuracy_score(y_train, y_train_model)))
+
+    y_pred = ensemble_model.predict(X_test)
+    print("**Test :** " + str(accuracy_score(y_test, y_pred)))
+
+    joblib.dump(ensemble_model, output_model_folder + '/' +  p_output + '.joblib') 
+
+if __name__== "__main__":
+    main()

svm_model_train.py

@@ -13,9 +13,11 @@ from sklearn.metrics import accuracy_score
 
 import sys, os, getopt
 
-output_model_folder = './saved_models/'
+current_dirpath = os.getcwd()
+output_model_folder = os.path.join(current_dirpath, 'saved_models')
 
 def get_best_model(X_train, y_train):
+    
     parameters = {'kernel':['rbf'], 'C': np.arange(1, 20)}
     svc = svm.SVC(gamma="scale")
     clf = GridSearchCV(svc, parameters, cv=5, scoring='accuracy', verbose=10)
@@ -82,7 +84,7 @@ def main():
     y_pred = svm_model.predict(X_test)
     print("**Test :** " + str(accuracy_score(y_test, y_pred)))
 
-    joblib.dump(svm_model, output_model_folder + p_output + '.joblib') 
+    joblib.dump(svm_model, output_model_folder + '/' + p_output + '.joblib') 
 
 if __name__== "__main__":
     main()

predict_noisy_image_sdv_lab.py

@@ -34,7 +34,7 @@ def main():
         elif o in ("-o", "--mode"):
             p_mode = a
 
-            if p_mode != 'svdn' and p_mode != 'svdne':
+            if p_mode != 'svdn' and p_mode != 'svdne' and p_mode != 'svd':
                 assert False, "Mode not recognized"
         else:
             assert False, "unhandled option"
@@ -55,15 +55,17 @@ def main():
             min = float(f.readline().replace('\n', ''))
             max = float(f.readline().replace('\n', ''))
 
-        l_values_normalized = image_processing.normalize_arr_with_range(LAB_L, min, max)
+        l_values = image_processing.normalize_arr_with_range(LAB_L, min, max)
 
+    elif p_mode == 'svdn':
+        l_values = image_processing.normalize_arr(LAB_L)
     else:
-        l_values_normalized = image_processing.normalize_arr(LAB_L)
+        l_values = LAB_L
 
     
     # get interval values
     begin, end = p_interval
-    test_data = l_values_normalized[begin:end]
+    test_data = l_values[begin:end]
 
     # get prediction of model
     prediction = model.predict([test_data])[0]

predict_seuil_expe.py

@@ -7,29 +7,36 @@ from PIL import Image
 
 import sys, os, getopt
 import subprocess
+import time
+
+current_dirpath = os.getcwd()
 
 config_filename   = "config"
 scenes_path = './fichiersSVD_light'
 min_max_filename = 'min_max_values'
-seuil_expe_filename = 'seuilExpe'
-tmp_filename = '/tmp/img_to_predict.png'
+threshold_expe_filename = 'seuilExpe'
+tmp_filename = '/tmp/__model__img_to_predict.png'
+
+threshold_map_folder = "threshold_map"
+threshold_map_file_prefix = "treshold_map_"
+
 zones = np.arange(16)
 
 def main():
 
     if len(sys.argv) <= 1:
         print('Run with default parameters...')
-        print('python predict_noisy_image.py --interval "0,20" --model path/to/xxxx.joblib --mode ["svdn", "svdne"]')
+        print('python predict_noisy_image.py --interval "0,20" --model path/to/xxxx.joblib --mode ["svdn", "svdne"] --limit_detection xx')
         sys.exit(2)
     try:
-        opts, args = getopt.getopt(sys.argv[1:], "ht:m:o", ["help=", "interval=", "model=", "mode="])
+        opts, args = getopt.getopt(sys.argv[1:], "ht:m:o:l", ["help=", "interval=", "model=", "mode=", "limit_detection="])
     except getopt.GetoptError:
         # print help information and exit:
-        print('python predict_noisy_image.py --interval "xx,xx" --model path/to/xxxx.joblib --mode ["svdn", "svdne"]')
+        print('python predict_noisy_image.py --interval "xx,xx" --model path/to/xxxx.joblib --mode ["svdn", "svdne"] --limit_detection xx')
         sys.exit(2)
     for o, a in opts:
         if o == "-h":
-            print('python predict_noisy_image.py --interval "xx,xx" --model path/to/xxxx.joblib --mode ["svdn", "svdne"]')
+            print('python predict_noisy_image.py --interval "xx,xx" --model path/to/xxxx.joblib --mode ["svdn", "svdne"] --limit_detection xx')
             sys.exit()
         elif o in ("-t", "--interval"):
             p_interval = a
@@ -38,8 +45,11 @@ def main():
         elif o in ("-o", "--mode"):
             p_mode = a
 
-            if p_mode != 'svdn' and p_mode != 'svdne':
+            if p_mode != 'svdn' and p_mode != 'svdne' and p_mode != 'svd':
                 assert False, "Mode not recognized"
+    
+        elif o in ("-l", "--limit_detection"):
+            p_limit = int(a)
         else:
             assert False, "unhandled option"
 
@@ -60,10 +70,10 @@ def main():
             end_index_image = config_file.readline().strip()
             step_counter = int(config_file.readline().strip())
 
-        seuil_expes = []
-        seuil_expes_detected = []
-        seuil_expes_counter = []
-        seuil_expes_found = []
+        threshold_expes = []
+        threshold_expes_detected = []
+        threshold_expes_counter = []
+        threshold_expes_found = []
 
         # get zones list info
         for index in zones:
@@ -72,22 +82,22 @@ def main():
                 index_str = "0" + index_str
             zone_folder = "zone"+index_str
 
-            with open(scene_path + "/" + zone_folder + "/" + seuil_expe_filename) as f:
-                seuil_expes.append(int(f.readline()))
-
-                # Initialize default data to get detected model seuil found
-                seuil_expes_detected.append(False)
-                seuil_expes_counter.append(0)
-                seuil_expes_found.append(0)
+            with open(scene_path + "/" + zone_folder + "/" + threshold_expe_filename) as f:
+                threshold = int(f.readline())
+                threshold_expes.append(threshold)
 
-        for seuil in seuil_expes:
-            print(seuil)
+                # Initialize default data to get detected model threshold found
+                threshold_expes_detected.append(False)
+                threshold_expes_counter.append(0)
+                threshold_expes_found.append(int(end_index_image)) # by default use max
 
         current_counter_index = int(start_index_image)
         end_counter_index = int(end_index_image)
 
         print(current_counter_index)
-        while(current_counter_index <= end_counter_index):
+        check_all_done = False
+
+        while(current_counter_index <= end_counter_index and not check_all_done):
             
             current_counter_index_str = str(current_counter_index)
 
@@ -96,37 +106,99 @@ def main():
 
             img_path = scene_path + "/" + prefix_image_name + current_counter_index_str + ".png"
 
-            print(img_path)
-
             current_img = Image.open(img_path)
             img_blocks = image_processing.divide_in_blocks(current_img, (200, 200))
 
+
+            check_all_done = all(d == True for d in threshold_expes_detected)
+
             for id_block, block in enumerate(img_blocks):
-                block.save(tmp_filename)
-
-                python_cmd = "python predict_noisy_image_sdv_lab.py --image " + tmp_filename + \
-                                   " --interval '" + p_interval + \
-                                   "' --model " + p_model_file  + \
-                                   " --mode " + p_mode
-                ## call command ##
-                p = subprocess.Popen(python_cmd, stdout=subprocess.PIPE, shell=True)
                 
-                (output, err) = p.communicate()
-                
-                ## Wait for result ##
-                p_status = p.wait()
+                # check only if necessary for this scene (not already detected)
+                if not threshold_expes_detected[id_block]:
+                    tmp_file_path = tmp_filename.replace('__model__', '_' + p_model_file.split('/')[1])
+                    block.save(tmp_file_path)
+
+                    python_cmd = "python predict_noisy_image_sdv_lab.py --image " + tmp_file_path + \
+                                    " --interval '" + p_interval + \
+                                    "' --model " + p_model_file  + \
+                                    " --mode " + p_mode
+                    ## call command ##
+                    p = subprocess.Popen(python_cmd, stdout=subprocess.PIPE, shell=True)
+                    
+                    (output, err) = p.communicate()
+                    
+                    ## Wait for result ##
+                    p_status = p.wait()
+
+                    prediction = int(output)
+
+                    if prediction == 0:
+                        threshold_expes_counter[id_block] = threshold_expes_counter[id_block] + 1
+                    else:
+                        threshold_expes_counter[id_block] = 0
+                    
+                    if threshold_expes_counter[id_block] == p_limit:
+                        threshold_expes_detected[id_block] = True
+                        threshold_expes_found[id_block] = current_counter_index
+
+                    print(str(id_block) + " : " + str(current_counter_index) + "/" + str(threshold_expes[id_block]) + " => " + str(prediction))
 
-                prediction = int(output)
+            current_counter_index += step_counter
+            print("------------------------")
+            print("Scene " + str(id_scene + 1) + "/" + str(len(scenes)))
+            print("------------------------")
 
+        # end of scene => display of results
 
+        model_treshold_path = threshold_map_folder + '/' + p_model_file.split('/')[1]
+        if not os.path.exists(model_treshold_path):
+            os.makedirs(model_treshold_path)
 
-                print(str(current_counter_index) + "/" + str(seuil_expes[id_block]) + " => " + str(prediction))
+        abs_dist = []
 
-            current_counter_index += step_counter
+        map_filename = model_treshold_path + "/" + threshold_map_file_prefix + folder_scene
+        f_map = open(map_filename, 'w')
+
+        line_information = ""
+
+        # default header
+        f_map.write('|  |    |    |  |\n')
+        f_map.write('---|----|----|---\n')
+        for id, threshold in enumerate(threshold_expes_found):
+
+            line_information += str(threshold) + " / " + str(threshold_expes[id]) + " | "
+            abs_dist.append(abs(threshold - threshold_expes[id]))
+
+            if (id + 1) % 4 == 0:
+                f_map.write(line_information + '\n')
+                line_information = ""
         
-        # end of scene => display of results
+        f_map.write(line_information + '\n')
+
+        min_abs_dist = min(abs_dist)
+        max_abs_dist = max(abs_dist)
+        avg_abs_dist = sum(abs_dist) / len(abs_dist)
+
+        f_map.write('\nScene information : ')
+        f_map.write('\n- BEGIN : ' + str(start_index_image))
+        f_map.write('\n- END : ' + str(end_index_image))
+
+        f_map.write('\n\nDistances information : ')
+        f_map.write('\n- MIN : ' + str(min_abs_dist))
+        f_map.write('\n- MAX : ' + str(max_abs_dist))
+        f_map.write('\n- AVG : ' + str(avg_abs_dist))
+
+        f_map.write('\n\nOther information : ')
+        f_map.write('\n- Detection limit : ' + str(p_limit))
+
+        # by default print last line
+        f_map.close()
 
+        print("Scene " + str(id_scene + 1) + "/" + str(len(scenes)) + " Done..")
+        print("------------------------")
 
+        time.sleep(10)
 
 
 if __name__== "__main__":

prediction.py

@@ -13,17 +13,17 @@ def main():
 
     if len(sys.argv) <= 1:
         print('Run with default parameters...')
-        print('python smv_model_train.py --data xxxx.csv --model xxxx.joblib --output xxxx --scene xxxx')
+        print('python prediction_scene.py --data xxxx.csv --model xxxx.joblib --output xxxx --scene xxxx')
         sys.exit(2)
     try:
         opts, args = getopt.getopt(sys.argv[1:], "hd:o:s", ["help=", "data=", "model=", "output=", "scene="])
     except getopt.GetoptError:
         # print help information and exit:
-        print('python smv_model_train.py --data xxxx.csv --model xxxx.joblib --output xxxx --scene xxxx')
+        print('python prediction_scene.py --data xxxx.csv --model xxxx.joblib --output xxxx --scene xxxx')
         sys.exit(2)
     for o, a in opts:
         if o == "-h":
-            print('python smv_model_train.py --data xxxx.csv --model xxxx.joblib --output xxxx --scene xxxx')
+            print('python prediction_scene.py --data xxxx.csv --model xxxx.joblib --output xxxx --scene xxxx')
             sys.exit()
         elif o in ("-d", "--data"):
             p_data_file = a

save_model_result_in_md.py

@@ -0,0 +1,98 @@
+from sklearn.externals import joblib
+
+import numpy as np
+
+from ipfml import image_processing
+from PIL import Image
+
+import sys, os, getopt
+import subprocess
+import time
+
+current_dirpath = os.getcwd()
+
+threshold_map_folder = "threshold_map"
+threshold_map_file_prefix = "treshold_map_"
+
+markdowns_folder = "models_info"
+
+zones = np.arange(16)
+
+def main():
+
+    if len(sys.argv) <= 1:
+        print('Run with default parameters...')
+        print('python save_model_result_in_md.py --interval "0,20" --model path/to/xxxx.joblib --mode ["svdn", "svdne"]')
+        sys.exit(2)
+    try:
+        opts, args = getopt.getopt(sys.argv[1:], "ht:m:o:l", ["help=", "interval=", "model=", "mode="])
+    except getopt.GetoptError:
+        # print help information and exit:
+        print('python save_model_result_in_md.py --interval "xx,xx" --model path/to/xxxx.joblib --mode ["svdn", "svdne"]')
+        sys.exit(2)
+    for o, a in opts:
+        if o == "-h":
+            print('python save_model_result_in_md.py --interval "xx,xx" --model path/to/xxxx.joblib --mode ["svdn", "svdne"]')
+            sys.exit()
+        elif o in ("-t", "--interval"):
+            p_interval = list(map(int, a.split(',')))
+        elif o in ("-m", "--model"):
+            p_model_file = a
+        elif o in ("-o", "--mode"):
+            p_mode = a
+
+            if p_mode != 'svdn' and p_mode != 'svdne' and p_mode != 'svd':
+                assert False, "Mode not recognized"
+        else:
+            assert False, "unhandled option"
+
+    
+    # call model and get global result in scenes
+
+    begin, end = p_interval
+
+    bash_cmd = "bash testModelByScene.sh '" + str(begin) + "' '" + str(end) + "' '" + p_model_file + "' '" + p_mode + "'"
+    print(bash_cmd)
+     
+    ## call command ##
+    p = subprocess.Popen(bash_cmd, stdout=subprocess.PIPE, shell=True)
+    
+    (output, err) = p.communicate()
+    
+    ## Wait for result ##
+    p_status = p.wait()
+
+    if not os.path.exists(markdowns_folder):
+        os.makedirs(markdowns_folder)
+        
+    md_model_path = markdowns_folder + '/' + p_model_file.split('/')[1].replace('.joblib', '.md')
+
+    with open(md_model_path, 'w') as f:
+        f.write(output.decode("utf-8"))
+
+        # read each threshold_map information if exists
+        model_map_info_path = threshold_map_folder + '/' + p_model_file.replace('saved_models/', '')
+
+        if not os.path.exists(model_map_info_path):
+            f.write('\n\n No threshold map information')
+        else:
+            maps_files = os.listdir(model_map_info_path)
+
+            # get all map information
+            for t_map_file in maps_files:
+                
+                file_path = model_map_info_path + '/' + t_map_file
+                with open(file_path, 'r') as map_file:
+
+                    title_scene =  t_map_file.replace(threshold_map_file_prefix, '')
+                    f.write('\n\n## ' + title_scene + '\n')
+                    content = map_file.readlines()
+
+                    # getting each map line information
+                    for line in content:
+                        f.write(line)
+
+        f.close()
+
+if __name__== "__main__":
+    main()

testModelByScene.sh

@@ -48,6 +48,6 @@ for scene in {"A","B","C","D","E","F","G","H","I"}; do
 
   python generate_data_svm.py --output ${FILENAME} --interval "${INPUT_BEGIN},${INPUT_END}" --kind ${INPUT_MODE} --scenes "${scene}" --zones "${zones}" --percent 1 --sep ";" --rowindex "0"
 
-  python prediction.py --data "$FILENAME.train" --model ${INPUT_MODEL} --output "${INPUT_MODEL}_Scene${scene}_mode_${INPUT_MODE}.prediction" --scene ${scene}
+  python prediction_scene.py --data "$FILENAME.train" --model ${INPUT_MODEL} --output "${INPUT_MODEL}_Scene${scene}_mode_${INPUT_MODE}.prediction" --scene ${scene}
 
 done