Merge branch 'release/v0.2.5'

README.md

@@ -6,13 +6,13 @@
 pip install -r requirements.txt
 ```
 
-Generate all needed data for each metrics (which equires the the whole dataset. In order to get it, you need to contact us.
+Generate all needed data for each metrics (which requires the the whole dataset. In order to get it, you need to contact us).
 
 ```bash
 python generate_all_data.py --metric all
 ```
 
-For noise detection, many metrics are available :
+For noise detection, many metrics are available:
 - lab
 - mscn
 - mscn_revisited
@@ -22,51 +22,51 @@ For noise detection, many metrics are available :
 - low_bits_6
 - low_bits_4_shifted_2
 
-You can also specify metric you want to compute and image step to avoid some images :
+You can also specify metric you want to compute and image step to avoid some images:
 ```bash
 python generate_all_data.py --metric mscn --step 50
 ```
 
-- **step** : keep only image if image id % 50 == 0 (assumption is that keeping spaced data will let model better fit).
+- **step**: keep only image if image id % 50 == 0 (assumption is that keeping spaced data will let model better fit).
 
 ## How to use
 
-### Multiple folders and scripts are availables :
+### Multiple directories and scripts are available:
 
 
-- **fichiersSVD_light/\*** : all scene files information (zones of each scene, SVD descriptor files information and so on...).
-- **train_model.py** : script which is used to run specific model available.
-- **data/\*** : folder which will contain all *.train* & *.test* files in order to train model.
-- **saved_models/*.joblib** : all scikit learn models saved.
-- **models_info/*** : all markdown files generated to get quick information about model performance and prediction. This folder contains also **model_comparisons.csv** obtained after running runAll_maxwell.sh script.
-- **modules/\*** : contains all modules usefull for the whole project (such as configuration variables)
+- **fichiersSVD_light/\***: all scene files information (zones of each scene, SVD descriptor files information and so on...).
+- **train_model.py**: script which is used to run specific model available.
+- **data/\***: folder which will contain all *.train* & *.test* files in order to train model.
+- **saved_models/*.joblib**: all scikit learn models saved.
+- **models_info/***: all markdown files generated to get quick information about model performance and prediction. This folder contains also **model_comparisons.csv** obtained after running runAll_maxwell.sh script.
+- **modules/\***: contains all modules usefull for the whole project (such as configuration variables)
 
 ### Scripts for generating data files
 
-Two scripts can be used for generating data in order to fit model :
-- **generate_data_model.py** : zones are specified and stayed fixed for each scene
-- **generate_data_model_random.py** : zones are chosen randomly (just a number of zone is specified)
-- **generate_data_model_random_maxwell.py** : zones are chosen randomly (just a number of zone is specified). Only maxwell scene are used.
+Two scripts can be used for generating data in order to fit model:
+- **generate_data_model.py**: zones are specified and stayed fixed for each scene
+- **generate_data_model_random.py**: zones are chosen randomly (just a number of zone is specified)
+- **generate_data_model_random_maxwell.py**: zones are chosen randomly (just a number of zone is specified). Only maxwell scene are used.
 
 
-**Remark** : Note here that all python script have *--help* command.
+**Remark**: Note here that all python script have *--help* command.
 
 ```
 python generate_data_model.py --help
 
-python generate_data_model.py --output xxxx --interval 0,20  --kind svdne --scenes "A, B, D" --zones "0, 1, 2" --percent 0.7 --sep : --rowindex 1 --custom custom_min_max_filename
+python generate_data_model.py --output xxxx --interval 0,20  --kind svdne --scenes "A, B, D" --zones "0, 1, 2" --percent 0.7 --sep: --rowindex 1 --custom custom_min_max_filename
 ```
 
-Parameters explained :
-- **output** : filename of data (which will be split into two parts, *.train* and *.test* relative to your choices).
-- **interval** : the interval of data you want to use from SVD vector.
-- **kind** : kind of data ['svd', 'svdn', 'svdne']; not normalize, normalize vector only and normalize together.
-- **scenes** : scenes choice for training dataset.
-- **zones** : zones to take for training dataset.
-- **percent** : percent of data amount of zone to take (choose randomly) of zone
-- **sep** : output csv file seperator used
-- **rowindex** : if 1 then row will be like that 1:xxxxx, 2:xxxxxx, ..., n:xxxxxx
-- **custom** : specify if you want your data normalized using interval and not the whole singular values vector. If it is, the value of this parameter is the output filename which will store the min and max value found. This file will be usefull later to make prediction with model (optional parameter).
+Parameters explained:
+- **output**: filename of data (which will be split into two parts, *.train* and *.test* relative to your choices).
+- **interval**: the interval of data you want to use from SVD vector.
+- **kind**: kind of data ['svd', 'svdn', 'svdne']; not normalize, normalize vector only and normalize together.
+- **scenes**: scenes choice for training dataset.
+- **zones**: zones to take for training dataset.
+- **percent**: percent of data amount of zone to take (choose randomly) of zone
+- **sep**: output csv file seperator used
+- **rowindex**: if 1 then row will be like that 1:xxxxx, 2:xxxxxx, ..., n:xxxxxx
+- **custom**: specify if you want your data normalized using interval and not the whole singular values vector. If it is, the value of this parameter is the output filename which will store the min and max value found. This file will be usefull later to make prediction with model (optional parameter).
 
 ### Train model
 
@@ -80,31 +80,31 @@ Expected values for the **choice** parameter are ['svm_model', 'ensemble_model',
 
 ### Predict image using model
 
-Now we have a model trained, we can use it with an image as input :
+Now we have a model trained, we can use it with an image as input:
 
 ```bash
 python predict_noisy_image_svd.py --image path/to/image.png --interval "x,x" --model saved_models/xxxxxx.joblib --metric 'lab' --mode 'svdn' --custom 'min_max_filename'
 ```
 
-- **metric** : metric choice need to be one of the listed above.
-- **custom** : specify filename with custom min and max from your data interval. This file was generated using **custom** parameter of one of the **generate_data_model\*.py** script (optional parameter).
+- **metric**: metric choice need to be one of the listed above.
+- **custom**: specify filename with custom min and max from your data interval. This file was generated using **custom** parameter of one of the **generate_data_model\*.py** script (optional parameter).
 
-The model will return only 0 or 1 :
+The model will return only 0 or 1:
 - 1 means noisy image is detected.
 - 0 means image seem to be not noisy.
 
-All SVD metrics developed need :
+All SVD metrics developed need:
 - Name added into *metric_choices_labels* global array variable of **modules/utils/config.py** file.
 - A specification of how you compute the metric into *get_svd_data* method of **modules/utils/data_type.py** file.
 
 ### Predict scene using model
 
-Now we have a model trained, we can use it with an image as input :
+Now we have a model trained, we can use it with an image as input:
 
 ```bash
 python prediction_scene.py --data path/to/xxxx.csv --model saved_model/xxxx.joblib --output xxxxx --scene xxxx
 ```
-**Remark** : *scene* parameter expected need to be the correct name of the Scene.
+**Remark**: *scene* parameter expected need to be the correct name of the Scene.
 
 ### Visualize data
 
@@ -122,18 +122,18 @@ Once you have simulation done. Checkout your **threshold_map/%MODEL_NAME%/simula
 
 ### Test model on all scene data
 
-In order to see if a model well generalized, a bash script is available :
+In order to see if a model well generalized, a bash script is available:
 
 ```bash
 bash testModelByScene.sh '100' '110' 'saved_models/xxxx.joblib' 'svdne' 'lab'
 ```
 
-Parameters list :
-- 1 : Begin of interval of data from SVD to use
-- 2 : End of interval of data from SVD to use
-- 3 : Model we want to test
-- 4 : Kind of data input used by trained model
-- 5 : Metric used by model
+Parameters list:
+- 1: Begin of interval of data from SVD to use
+- 2: End of interval of data from SVD to use
+- 3: Model we want to test
+- 4: Kind of data input used by trained model
+- 5: Metric used by model
 
 
 ### Get treshold map
@@ -144,18 +144,18 @@ Main objective of this project is to predict as well as a human the noise percep
 python predict_seuil_expe.py --interval "x,x" --model 'saved_models/xxxx.joblib' --mode ["svd", "svdn", "svdne"] --metric ['lab', 'mscn', ...] --limit_detection xx --custom 'custom_min_max_filename'
 ```
 
-Parameters list :
-- **model** : mode file saved to use
-- **interval** : the interval of data you want to use from SVD vector.
-- **mode** : kind of data ['svd', 'svdn', 'svdne']; not normalize, normalize vector only and normalize together.
-- **limit_detection** : number of not noisy images found to stop and return threshold (integer).
-- **custom** : custom filename where min and max values are stored (optional parameter).
+Parameters list:
+- **model**: mode file saved to use
+- **interval**: the interval of data you want to use from SVD vector.
+- **mode**: kind of data ['svd', 'svdn', 'svdne']; not normalize, normalize vector only and normalize together.
+- **limit_detection**: number of not noisy images found to stop and return threshold (integer).
+- **custom**: custom filename where min and max values are stored (optional parameter).
 
 ### Display model performance information
 
 Another script was developed to display into Mardown format the performance of a model.
 
-The content will be divised into two parts :
+The content will be divised into two parts:
 - Predicted performance on all scenes
 - Treshold maps obtained from model on each scenes
 
@@ -165,10 +165,10 @@ The previous script need to already have ran to obtain and display treshold maps
 python save_model_result_in_md.py --interval "xx,xx" --model saved_models/xxxx.joblib --mode ["svd", "svdn", "svdne"] --metric ['lab', 'mscn']
 ```
 
-Parameters list :
-- **model** : mode file saved to use
-- **interval** : the interval of data you want to use from SVD vector.
-- **mode** : kind of data ['svd', 'svdn', 'svdne']; not normalize, normalize vector only and normalize together.
+Parameters list:
+- **model**: mode file saved to use
+- **interval**: the interval of data you want to use from SVD vector.
+- **mode**: kind of data ['svd', 'svdn', 'svdne']; not normalize, normalize vector only and normalize together.
 
 Markdown file with all information is saved using model name into **models_info** folder.
 

display_bits_shifted_scene.py

@@ -7,7 +7,7 @@ Created on Fri Sep 14 21:02:42 2018
 """
 
 from __future__ import print_function
-import sys, os, getopt
+import sys, os, argparse
 import numpy as np
 import random
 import time
@@ -28,7 +28,7 @@ min_max_filename    = cfg.min_max_filename_extension
 
 # define all scenes values
 scenes_list         = cfg.scenes_names
-scenes_indexes      = cfg.scenes_indices
+scenes_indices      = cfg.scenes_indices
 choices             = cfg.normalization_choices
 path                = cfg.dataset_path
 zones               = cfg.zones_indices
@@ -157,33 +157,15 @@ def display_data_scenes(nb_bits, p_scene):
 
 def main():
 
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python generate_all_data.py --bits 3 --scene A')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hb:s", ["help=", "bits=", "scene="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python generate_all_data.py --bits 4 --scene A')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python generate_all_data.py --bits 4 --scene A')
-            sys.exit()
-        elif o in ("-b", "--bits"):
-            p_bits = int(a)
-
-        elif o in ("-s", "--scene"):
-            p_scene = a
-
-            if p_scene not in scenes_indexes:
-                assert False, "Invalid metric choice"
-            else:
-                    p_scene = scenes_list[scenes_indexes.index(p_scene)]
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Display curves of shifted bits influence of L canal on specific scene")
 
+    parser.add_argument('--bits', type=str, help='Number of bits to display')
+    parser.add_argument('--scene', type=str, help="scene index to use", choices=scenes_indices)
+
+    args = parser.parse_args()
+
+    p_bits  = args.bits
+    p_scene = scenes_list[scenes_indices.index(args.scene)]
 
     display_data_scenes(p_bits, p_scene)
 

display_scenes_zones.py

@@ -7,7 +7,7 @@ Created on Fri Sep 14 21:02:42 2018
 """
 
 from __future__ import print_function
-import sys, os, getopt
+import sys, os, argparse
 import numpy as np
 import random
 import time
@@ -26,8 +26,8 @@ min_max_filename    = cfg.min_max_filename_extension
 
 # define all scenes values
 scenes_list         = cfg.scenes_names
-scenes_indexes      = cfg.scenes_indices
-choices             = cfg.normalization_choices
+scenes_indices      = cfg.scenes_indices
+norm_choices        = cfg.normalization_choices
 path                = cfg.dataset_path
 zones               = cfg.zones_indices
 seuil_expe_filename = cfg.seuil_expe_filename
@@ -145,7 +145,7 @@ def display_data_scenes(data_type, p_scene, p_kind):
                         # extract from temp image
                         data = metrics.get_SVD_s(img_block)
 
-                    if data_type == 'mscn':
+                    '''if data_type == 'mscn':
 
                         img_gray = np.array(color.rgb2gray(np.asarray(block))*255, 'uint8')
                         img_mscn = processing.calculate_mscn_coefficients(img_gray, 7)
@@ -153,7 +153,7 @@ def display_data_scenes(data_type, p_scene, p_kind):
 
                         img_mscn_gray = np.array(img_mscn_norm*255, 'uint8')
 
-                        data = metrics.get_SVD_s(img_mscn_gray)
+                        data = metrics.get_SVD_s(img_mscn_gray)'''
 
                     if data_type == 'low_bits_6':
 
@@ -231,40 +231,17 @@ def display_data_scenes(data_type, p_scene, p_kind):
 
 def main():
 
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python generate_all_data.py --metric all --scene A --kind svdn')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hm:s:k", ["help=", "metric=", "scene=", "kind="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python generate_all_data.py --metric all --scene A --kind svdn')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python generate_all_data.py --metric all --scene A --kind svdn')
-            sys.exit()
-        elif o in ("-m", "--metric"):
-            p_metric = a
-
-            if p_metric != 'all' and p_metric not in metric_choices:
-                assert False, "Invalid metric choice"
-        elif o in ("-s", "--scene"):
-            p_scene = a
-
-            if p_scene not in scenes_indexes:
-                assert False, "Invalid metric choice"
-            else:
-                p_scene = scenes_list[scenes_indexes.index(p_scene)]
-        elif o in ("-k", "--kind"):
-            p_kind = a
-
-            if p_kind not in choices:
-                assert False, "Invalid metric choice"
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Display zones curves of metric on scene ")
 
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=metric_choices)
+    parser.add_argument('--scene', type=str, help='scene index to use', choices=scenes_indices)
+    parser.add_argument('--kind', type=str, help='Kind of normalization level wished', choices=norm_choices)
+
+    args = parser.parse_args()
+
+    p_metric = args.metric
+    p_kind   = args.kind
+    p_scene  = scenes_list[scenes_indices.index(args.scene)]
 
     display_data_scenes(p_metric, p_scene, p_kind)
 

display_scenes_zones_shifted.py

@@ -7,7 +7,7 @@ Created on Fri Sep 14 21:02:42 2018
 """
 
 from __future__ import print_function
-import sys, os, getopt
+import sys, os, argparse
 import numpy as np
 import random
 import time
@@ -26,7 +26,7 @@ min_max_filename    = cfg.min_max_filename_extension
 
 # define all scenes values
 scenes_list         = cfg.scenes_names
-scenes_indexes      = cfg.scenes_indices
+scenes_indices      = cfg.scenes_indices
 choices             = cfg.normalization_choices
 path                = cfg.dataset_path
 zones               = cfg.zones_indices
@@ -155,35 +155,18 @@ def display_data_scenes(p_scene, p_bits, p_shifted):
             plt.show()
 
 def main():
+    
+    parser = argparse.ArgumentParser(description="Display curves of shifted bits influence of L canal on specific scene by zone")
 
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python generate_all_data.py --scene A --bits 3 --shifted 3')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hs:b:s", ["help=", "scene=", "bits=", "shifted="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python generate_all_data.py --scene A --bits 3 --shifted 3')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python generate_all_data.py --scene A --bits 3 --shifted 3')
-            sys.exit()
-        elif o in ("-b", "--bits"):
-            p_bits = int(a)
-
-        elif o in ("-s", "--scene"):
-            p_scene = a
-
-            if p_scene not in scenes_indexes:
-                assert False, "Invalid metric choice"
-            else:
-                p_scene = scenes_list[scenes_indexes.index(p_scene)]
-        elif o in ("-f", "--shifted"):
-            p_shifted = int(a)
-        else:
-            assert False, "unhandled option"
+    parser.add_argument('--scene', type=str, help='scene index to use', choices=scenes_indices)
+    parser.add_argument('--bits', type=str, help='Number of bits to used')
+    parser.add_argument('--shifted', type=str, help='Number of bits shifted')    
+
+    args = parser.parse_args()
+
+    p_scene   = scenes_list[scenes_indices.index(args.scene)]
+    p_bits    = args.bits
+    p_shifted = args.shifted
 
     if p_bits + p_shifted > max_nb_bits:
         assert False, "Invalid parameters, cannot have bits greater than 8 after shift move"

display_svd_area_data_scene.py

@@ -7,7 +7,7 @@ Created on Fri Sep 14 21:02:42 2018
 """
 
 from __future__ import print_function
-import sys, os, getopt
+import sys, os, argparse
 
 import numpy as np
 import random
@@ -244,67 +244,29 @@ def display_svd_values(p_scene, p_interval, p_indices, p_metric, p_mode, p_step,
 
 def main():
 
-
-    # by default p_step value is 10 to enable all photos
-    p_step = 10
-    p_ylim = (0, 1)
-
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python display_svd_area_data_scene.py --scene A --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --area simps --ylim "0, 0.1"')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hs:i:i:z:l:m:s:n:a:y", ["help=", "scene=", "interval=", "indices=", "metric=", "mode=", "step=", "norm=", "area=", "ylim="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python display_svd_area_data_scene.py --scene A --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --area simps --ylim "0, 0.1"')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python display_svd_area_data_scene.py --scene A --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --area simps --ylim "0, 0.1"')
-            sys.exit()
-        elif o in ("-s", "--scene"):
-            p_scene = a
-
-            if p_scene not in scenes_indices:
-                assert False, "Invalid scene choice"
-            else:
-                p_scene = scenes_list[scenes_indices.index(p_scene)]
-        elif o in ("-i", "--interval"):
-            p_interval = list(map(int, a.split(',')))
-
-        elif o in ("-i", "--indices"):
-            p_indices = list(map(int, a.split(',')))
-
-        elif o in ("-m", "--metric"):
-            p_metric = a
-
-            if p_metric not in metric_choices:
-                assert False, "Invalid metric choice"
-
-        elif o in ("-m", "--mode"):
-            p_mode = a
-
-            if p_mode not in choices:
-                assert False, "Invalid normalization choice, expected ['svd', 'svdn', 'svdne']"
-
-        elif o in ("-s", "--step"):
-            p_step = int(a)
-
-        elif o in ("-n", "--norm"):
-            p_norm = int(a)
-
-        elif o in ("-a", "--area"):
-            p_area = a
-
-            if p_area not in integral_area_choices:
-                assert False, "Invalid area computation choices : %s " % integral_area_choices
-
-        elif o in ("-y", "--ylim"):
-            p_ylim = list(map(float, a.split(',')))
-
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Display area under curve data on scene")
+
+    parser.add_argument('--scene', type=str, help='scene index to use', choices=cfg.scenes_indices)
+    parser.add_argument('--interval', type=str, help='Interval value to keep from svd', default='"0, 200"')
+    parser.add_argument('--indices', type=str, help='Samples interval to display', default='"0, 900"')
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=metric_choices)
+    parser.add_argument('--mode', type=str, help='Kind of normalization level wished', choices=cfg.normalization_choices)
+    parser.add_argument('--step', type=int, help='Each step samples to display', default=10)
+    parser.add_argument('--norm', type=int, help='If values will be normalized or not', choices=[0, 1])
+    parser.add_argument('--area', type=int, help='Way of computing area under curve', choices=integral_area_choices)
+    parser.add_argument('--ylim', type=str, help='ylim interval to use', default='"0, 1"')
+
+    args = parser.parse_args()
+
+    p_scene    = scenes_list[scenes_indices.index(args.scene)]
+    p_indices  = list(map(int, args.indices.split(',')))
+    p_interval = list(map(int, args.interval.split(',')))
+    p_metric   = args.metric
+    p_mode     = args.mode
+    p_step     = args.step
+    p_norm     = args.norm
+    p_area     = args.area
+    p_ylim     = list(map(int, args.ylim.split(',')))
 
     display_svd_values(p_scene, p_interval, p_indices, p_metric, p_mode, p_step, p_norm, p_area, p_ylim)
 

display_svd_area_scenes.py

@@ -7,7 +7,7 @@ Created on Fri Sep 14 21:02:42 2018
 """
 
 from __future__ import print_function
-import sys, os, getopt
+import sys, os, argparse
 
 import numpy as np
 import random
@@ -249,60 +249,29 @@ def display_svd_values(p_interval, p_indices, p_metric, p_mode, p_step, p_norm,
 
 def main():
 
-
-    # by default p_step value is 10 to enable all photos
-    p_step = 10
-    p_ylim = (0, 1)
-
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python display_svd_area_scenes.py --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --area simps --ylim "0, 0.1"')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hs:i:i:z:l:m:s:n:a:y", ["help=", "scene=", "interval=", "indices=", "metric=", "mode=", "step=", "norm=", "area=", "ylim="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python display_svd_area_scenes.py --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --area simps --ylim "0, 0.1"')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python display_svd_area_scenes.py --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --area simps --ylim "0, 0.1"')
-            sys.exit()
-        elif o in ("-i", "--interval"):
-            p_interval = list(map(int, a.split(',')))
-
-        elif o in ("-i", "--indices"):
-            p_indices = list(map(int, a.split(',')))
-
-        elif o in ("-m", "--metric"):
-            p_metric = a
-
-            if p_metric not in metric_choices:
-                assert False, "Invalid metric choice"
-
-        elif o in ("-m", "--mode"):
-            p_mode = a
-
-            if p_mode not in choices:
-                assert False, "Invalid normalization choice, expected ['svd', 'svdn', 'svdne']"
-
-        elif o in ("-s", "--step"):
-            p_step = int(a)
-
-        elif o in ("-n", "--norm"):
-            p_norm = int(a)
-
-        elif o in ("-a", "--area"):
-            p_area = a
-
-            if p_area not in integral_area_choices:
-                assert False, "Invalid area computation choices : %s " % integral_area_choices
-
-        elif o in ("-y", "--ylim"):
-            p_ylim = list(map(float, a.split(',')))
-
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Display area under curve on scene")
+
+    parser.add_argument('--scene', type=str, help='scene index to use', choices=cfg.scenes_indices)
+    parser.add_argument('--interval', type=str, help='Interval value to keep from svd', default='"0, 200"')
+    parser.add_argument('--indices', type=str, help='Samples interval to display', default='"0, 900"')
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=metric_choices)
+    parser.add_argument('--mode', type=str, help='Kind of normalization level wished', choices=cfg.normalization_choices)
+    parser.add_argument('--step', type=int, help='Each step samples to display', default=10)
+    parser.add_argument('--norm', type=int, help='If values will be normalized or not', choices=[0, 1])
+    parser.add_argument('--area', type=int, help='Way of computing area under curve', choices=integral_area_choices)
+    parser.add_argument('--ylim', type=str, help='ylim interval to use', default='"0, 1"')
+
+    args = parser.parse_args()
+
+    p_scene    = scenes_list[scenes_indices.index(args.scene)]
+    p_indices  = list(map(int, args.indices.split(',')))
+    p_interval = list(map(int, args.interval.split(',')))
+    p_metric   = args.metric
+    p_mode     = args.mode
+    p_step     = args.step
+    p_norm     = args.norm
+    p_area     = args.area
+    p_ylim     = list(map(int, args.ylim.split(',')))
 
     display_svd_values(p_interval, p_indices, p_metric, p_mode, p_step, p_norm, p_area, p_ylim)
 

display_svd_data_error_scene.py

@@ -7,7 +7,7 @@ Created on Fri Sep 14 21:02:42 2018
 """
 
 from __future__ import print_function
-import sys, os, getopt
+import sys, os, argparse
 
 import numpy as np
 import random
@@ -181,7 +181,6 @@ def display_svd_values(p_scene, p_interval, p_indices, p_metric, p_mode, p_step,
                 print('%.2f%%' % (current_counter_index / end_counter_index * 100))
                 sys.stdout.write("\033[F")
 
-
             # all indices of picture to plot
             print(images_indices)
 
@@ -205,7 +204,7 @@ def display_svd_values(p_scene, p_interval, p_indices, p_metric, p_mode, p_step,
 
                 # use of whole image data for computation of ssim or psnr
                 if p_error == 'ssim' or p_error == 'psnr':
-                    image_path = file_path.format(str(current_id))
+                    image_path = file_path.format(str(images_indices[id]))
                     current_data = np.asarray(Image.open(image_path))
 
                 if len(previous_data) > 0:
@@ -258,64 +257,29 @@ def display_svd_values(p_scene, p_interval, p_indices, p_metric, p_mode, p_step,
 
 def main():
 
-
-    # by default p_step value is 10 to enable all photos
-    p_step = 10
-    p_ylim = (0, 1)
-
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python display_svd_data_scene.py --scene A --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --error mae --ylim "0, 0.1"')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hs:i:i:z:l:m:s:n:e:y", ["help=", "scene=", "interval=", "indices=", "metric=", "mode=", "step=", "norm=", "error=", "ylim="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python display_svd_data_scene.py --scene A --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --error mae --ylim "0, 0.1"')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python display_svd_data_scene.py --scene A --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --error mae --ylim "0, 0.1"')
-            sys.exit()
-        elif o in ("-s", "--scene"):
-            p_scene = a
-
-            if p_scene not in scenes_indices:
-                assert False, "Invalid scene choice"
-            else:
-                p_scene = scenes_list[scenes_indices.index(p_scene)]
-        elif o in ("-i", "--interval"):
-            p_interval = list(map(int, a.split(',')))
-
-        elif o in ("-i", "--indices"):
-            p_indices = list(map(int, a.split(',')))
-
-        elif o in ("-m", "--metric"):
-            p_metric = a
-
-            if p_metric not in metric_choices:
-                assert False, "Invalid metric choice"
-
-        elif o in ("-m", "--mode"):
-            p_mode = a
-
-            if p_mode not in choices:
-                assert False, "Invalid normalization choice, expected ['svd', 'svdn', 'svdne']"
-
-        elif o in ("-s", "--step"):
-            p_step = int(a)
-
-        elif o in ("-n", "--norm"):
-            p_norm = int(a)
-
-        elif o in ("-e", "--error"):
-            p_error = a
-
-        elif o in ("-y", "--ylim"):
-            p_ylim = list(map(float, a.split(',')))
-
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Display evolution of error on scene")
+
+    parser.add_argument('--scene', type=str, help='scene index to use', choices=cfg.scenes_indices)
+    parser.add_argument('--interval', type=str, help='Interval value to keep from svd', default='"0, 200"')
+    parser.add_argument('--indices', type=str, help='Samples interval to display', default='"0, 900"')
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=metric_choices)
+    parser.add_argument('--mode', type=str, help='Kind of normalization level wished', choices=cfg.normalization_choices)
+    parser.add_argument('--step', type=int, help='Each step samples to display', default=10)
+    parser.add_argument('--norm', type=int, help='If values will be normalized or not', choices=[0, 1])
+    parser.add_argument('--error', type=int, help='Way of computing error', choices=error_data_choices)
+    parser.add_argument('--ylim', type=str, help='ylim interval to use', default='"0, 1"')
+
+    args = parser.parse_args()
+
+    p_scene    = scenes_list[scenes_indices.index(args.scene)]
+    p_indices  = list(map(int, args.indices.split(',')))
+    p_interval = list(map(int, args.interval.split(',')))
+    p_metric   = args.metric
+    p_mode     = args.mode
+    p_step     = args.step
+    p_norm     = args.norm
+    p_error    = args.error
+    p_ylim     = list(map(int, args.ylim.split(',')))
 
     display_svd_values(p_scene, p_interval, p_indices, p_metric, p_mode, p_step, p_norm, p_error, p_ylim)
 

display_svd_data_scene.py

@@ -7,7 +7,7 @@ Created on Fri Sep 14 21:02:42 2018
 """
 
 from __future__ import print_function
-import sys, os, getopt
+import sys, os, argparse
 
 import numpy as np
 import random
@@ -222,61 +222,27 @@ def display_svd_values(p_scene, p_interval, p_indices, p_metric, p_mode, p_step,
 
 def main():
 
-
-    # by default p_step value is 10 to enable all photos
-    p_step = 10
-    p_ylim = (0, 1)
-
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python display_svd_data_scene.py --scene A --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --ylim "0, 0.1"')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hs:i:i:z:l:m:s:n:e:y", ["help=", "scene=", "interval=", "indices=", "metric=", "mode=", "step=", "norm=", "error=", "ylim="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python display_svd_data_scene.py --scene A --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --ylim "0, 0.1"')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python display_svd_data_scene.py --scene A --interval "0,800" --indices "0, 900" --metric lab --mode svdne --step 50 --norm 0 --ylim "0, 0.1"')
-            sys.exit()
-        elif o in ("-s", "--scene"):
-            p_scene = a
-
-            if p_scene not in scenes_indices:
-                assert False, "Invalid scene choice"
-            else:
-                p_scene = scenes_list[scenes_indices.index(p_scene)]
-        elif o in ("-i", "--interval"):
-            p_interval = list(map(int, a.split(',')))
-
-        elif o in ("-i", "--indices"):
-            p_indices = list(map(int, a.split(',')))
-
-        elif o in ("-m", "--metric"):
-            p_metric = a
-
-            if p_metric not in metric_choices:
-                assert False, "Invalid metric choice"
-
-        elif o in ("-m", "--mode"):
-            p_mode = a
-
-            if p_mode not in choices:
-                assert False, "Invalid normalization choice, expected ['svd', 'svdn', 'svdne']"
-
-        elif o in ("-s", "--step"):
-            p_step = int(a)
-
-        elif o in ("-n", "--norm"):
-            p_norm = int(a)
-
-        elif o in ("-y", "--ylim"):
-            p_ylim = list(map(float, a.split(',')))
-
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Display SVD data of scene")
+
+    parser.add_argument('--scene', type=str, help='scene index to use', choices=cfg.scenes_indices)
+    parser.add_argument('--interval', type=str, help='Interval value to keep from svd', default='"0, 200"')
+    parser.add_argument('--indices', type=str, help='Samples interval to display', default='"0, 900"')
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=metric_choices)
+    parser.add_argument('--mode', type=str, help='Kind of normalization level wished', choices=cfg.normalization_choices)
+    parser.add_argument('--step', type=int, help='Each step samples to display', default=10)
+    parser.add_argument('--norm', type=int, help='If values will be normalized or not', choices=[0, 1])
+    parser.add_argument('--ylim', type=str, help='ylim interval to use', default='"0, 1"')
+
+    args = parser.parse_args()
+
+    p_scene    = scenes_list[scenes_indices.index(args.scene)]
+    p_indices  = list(map(int, args.indices.split(',')))
+    p_interval = list(map(int, args.interval.split(',')))
+    p_metric   = args.metric
+    p_mode     = args.mode
+    p_step     = args.step
+    p_norm     = args.norm
+    p_ylim     = list(map(int, args.ylim.split(',')))
 
     display_svd_values(p_scene, p_interval, p_indices, p_metric, p_mode, p_step, p_norm, p_ylim)
 

display_svd_zone_scene.py

@@ -7,7 +7,7 @@ Created on Fri Sep 14 21:02:42 2018
 """
 
 from __future__ import print_function
-import sys, os, getopt
+import sys, os, argparse
 
 import numpy as np
 import random
@@ -252,68 +252,29 @@ def display_svd_values(p_scene, p_interval, p_indices, p_zone, p_metric, p_mode,
 
 def main():
 
-
-    # by default p_step value is 10 to enable all photos
-    p_step = 10
-    p_norm = 0
-    p_ylim = (0, 1)
-
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python display_svd_zone_scene.py --scene A --interval "0,200" --indices "0, 900" --zone 3 --metric lab --mode svdne --step 50 --norm 0 --ylim "0, 0.1"')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hs:i:i:z:l:m:s:n:y", ["help=", "scene=", "interval=", "indices=", "zone=", "metric=", "mode=", "step=", "norm=", "ylim="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python display_svd_zone_scene.py --scene A --interval "0,200" --indices "0, 900" --zone 3 --metric lab --mode svdne --step 50 --norm 0 --ylim "0, 0.1"')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python display_svd_zone_scene.py --scene A --interval "0,200" --indices "0, 900" --zone 3 --metric lab --mode svdne --step 50 --norm 0 --ylim "0, 0.1"')
-            sys.exit()
-        elif o in ("-s", "--scene"):
-            p_scene = a
-
-            if p_scene not in scenes_indices:
-                assert False, "Invalid scene choice"
-            else:
-                p_scene = scenes_list[scenes_indices.index(p_scene)]
-        elif o in ("-i", "--interval"):
-            p_interval = list(map(int, a.split(',')))
-
-        elif o in ("-i", "--indices"):
-            p_indices = list(map(int, a.split(',')))
-
-        elif o in ("-z", "--zone"):
-            p_zone = int(a)
-
-        elif o in ("-m", "--metric"):
-            p_metric = a
-
-            if p_metric not in metric_choices:
-                assert False, "Invalid metric choice"
-
-        elif o in ("-m", "--mode"):
-            p_mode = a
-
-            if p_mode not in choices:
-                assert False, "Invalid normalization choice, expected ['svd', 'svdn', 'svdne']"
-
-        elif o in ("-s", "--step"):
-            p_step = int(a)
-
-        elif o in ("-n", "--norm"):
-            p_norm = int(a)
-
-        elif o in ("-y", "--ylim"):
-            p_ylim = list(map(float, a.split(',')))
-
-        else:
-            assert False, "unhandled option"
-
-    if p_norm:
-        get_min_max_value_interval(p_scene, p_interval, p_metric)
+    parser = argparse.ArgumentParser(description="Display SVD data of scene zone")
+
+    parser.add_argument('--scene', type=str, help='scene index to use', choices=cfg.scenes_indices)
+    parser.add_argument('--interval', type=str, help='Interval value to keep from svd', default='"0, 200"')
+    parser.add_argument('--indices', type=str, help='Samples interval to display', default='"0, 900"')
+    parser.add_argument('--zone', type=int, help='Zone to display', choices=list(range(0, 16)))
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=metric_choices)
+    parser.add_argument('--mode', type=str, help='Kind of normalization level wished', choices=cfg.normalization_choices)
+    parser.add_argument('--step', type=int, help='Each step samples to display', default=10)
+    parser.add_argument('--norm', type=int, help='If values will be normalized or not', choices=[0, 1])
+    parser.add_argument('--ylim', type=str, help='ylim interval to use', default='"0, 1"')
+
+    args = parser.parse_args()
+
+    p_scene    = scenes_list[scenes_indices.index(args.scene)]
+    p_indices  = list(map(int, args.indices.split(',')))
+    p_interval = list(map(int, args.interval.split(',')))
+    p_zone     = args.zone
+    p_metric   = args.metric
+    p_mode     = args.mode
+    p_step     = args.step
+    p_norm     = args.norm
+    p_ylim     = list(map(int, args.ylim.split(',')))
 
     display_svd_values(p_scene, p_interval, p_indices, p_zone, p_metric, p_mode, p_step, p_norm, p_ylim)
 

generate_all_data.py

@@ -7,7 +7,7 @@ Created on Fri Sep 14 21:02:42 2018
 """
 
 from __future__ import print_function
-import sys, os, getopt
+import sys, os, argparse
 import numpy as np
 import random
 import time
@@ -180,32 +180,15 @@ def generate_data_svd(data_type, mode):
 
 def main():
 
-    # default value of p_step
-    p_step = 1
-
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python generate_all_data.py --metric all')
-        print('python generate_all_data.py --metric lab')
-        print('python generate_all_data.py --metric lab')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hms", ["help=", "metric="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python generate_all_data.py --metric all')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python generate_all_data.py --metric all')
-            sys.exit()
-        elif o in ("-m", "--metric"):
-            p_metric = a
-
-            if p_metric != 'all' and p_metric not in metric_choices:
-                assert False, "Invalid metric choice"
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Compute and prepare data of metric of all scenes (keep in memory min and max value found)")
+
+    parser.add_argument('--metric', type=str, 
+                                    help="metric choice in order to compute data (use 'all' if all metrics are needed)", 
+                                    choices=metric_choices)
+
+    args = parser.parse_args()
+
+    p_metric = args.metric
 
     # generate all or specific metric data
     if p_metric == 'all':

predict_seuil_expe.py

@@ -5,7 +5,7 @@ import numpy as np
 from ipfml import processing, utils
 from PIL import Image
 
-import sys, os, getopt
+import sys, os, argparse
 import subprocess
 import time
 
@@ -20,6 +20,8 @@ threshold_map_folder      = cfg.threshold_map_folder
 threshold_map_file_prefix = cfg.threshold_map_folder + "_"
 
 zones                     = cfg.zones_indices
+normalization_choices     = cfg.normalization_choices
+metric_choices            = cfg.metric_choices_labels
 
 tmp_filename              = '/tmp/__model__img_to_predict.png'
 
@@ -29,41 +31,25 @@ def main():
 
     p_custom = False
 
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python predict_seuil_expe.py --interval "0,20" --model path/to/xxxx.joblib --mode svdn --metric lab --limit_detection xx --custom min_max_filename')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "ht:m:o:l:c", ["help=", "interval=", "model=", "mode=", "metric=" "limit_detection=", "custom="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python predict_seuil_expe.py --interval "xx,xx" --model path/to/xxxx.joblib --mode svdn --metric lab --limit_detection xx --custom min_max_filename')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python predict_seuil_expe.py --interval "xx,xx" --model path/to/xxxx.joblib --mode svdn --metric lab --limit_detection xx --custom min_max_filename')
-            sys.exit()
-        elif o in ("-t", "--interval"):
-            p_interval = a
-        elif o in ("-mo", "--model"):
-            p_model_file = a
-        elif o in ("-o", "--mode"):
-            p_mode = a
-
-            if p_mode != 'svdn' and p_mode != 'svdne' and p_mode != 'svd':
-                assert False, "Mode not recognized"
-
-        elif o in ("-me", "--metric"):
-            p_metric = a
-        elif o in ("-l", "--limit_detection"):
-            p_limit = int(a)
-        elif o in ("-c", "--custom"):
-            p_custom = a
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Script which predicts threshold using specific model")
 
-    scenes = os.listdir(scenes_path)
+    parser.add_argument('--interval', type=str, help='Interval value to keep from svd', default='"0, 200"')
+    parser.add_argument('--model', type=str, help='.joblib or .json file (sklearn or keras model)')
+    parser.add_argument('--mode', type=str, help='Kind of normalization level wished', choices=normalization_choices)
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=metric_choices)
+    #parser.add_argument('--limit_detection', type=int, help='Specify number of same prediction to stop threshold prediction', default=2)
+    parser.add_argument('--custom', type=str, help='Name of custom min max file if use of renormalization of data', default=False)
+
+    args = parser.parse_args()
 
+    p_interval   = list(map(int, args.interval.split(',')))
+    p_model_file = args.model
+    p_mode       = args.mode
+    p_metric     = args.metric
+    #p_limit      = args.limit
+    p_custom     = args.custom
+
+    scenes = os.listdir(scenes_path)
     scenes = [s for s in scenes if not min_max_filename in s]
 
     # go ahead each scenes

predict_seuil_expe_maxwell.py

@@ -5,7 +5,7 @@ import numpy as np
 from ipfml import processing
 from PIL import Image
 
-import sys, os, getopt
+import sys, os, argparse
 import subprocess
 import time
 
@@ -22,6 +22,8 @@ threshold_map_file_prefix = cfg.threshold_map_folder + "_"
 
 zones                     = cfg.zones_indices
 maxwell_scenes            = cfg.maxwell_scenes_names
+normalization_choices     = cfg.normalization_choices
+metric_choices            = cfg.metric_choices_labels
 
 tmp_filename              = '/tmp/__model__img_to_predict.png'
 
@@ -32,41 +34,25 @@ def main():
     # by default..
     p_custom = False
 
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python predict_seuil_expe_maxwell.py --interval "0,20" --model path/to/xxxx.joblib --mode svdn --metric lab --limit_detection xx --custom min_max_filename')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "ht:m:o:l:c", ["help=", "interval=", "model=", "mode=", "metric=", "limit_detection=", "custom="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python predict_seuil_expe_maxwell.py --interval "xx,xx" --model path/to/xxxx.joblib --mode svdn --metric lab --limit_detection xx --custom min_max_filename')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python predict_seuil_expe_maxwell.py --interval "xx,xx" --model path/to/xxxx.joblib --mode svdn --metric lab --limit_detection xx --custom min_max_filename')
-            sys.exit()
-        elif o in ("-t", "--interval"):
-            p_interval = a
-        elif o in ("-m", "--model"):
-            p_model_file = a
-        elif o in ("-o", "--mode"):
-            p_mode = a
-
-            if p_mode != 'svdn' and p_mode != 'svdne' and p_mode != 'svd':
-                assert False, "Mode not recognized"
-
-        elif o in ("-m", "--metric"):
-            p_metric = a
-        elif o in ("-l", "--limit_detection"):
-            p_limit = int(a)
-        elif o in ("-c", "--custom"):
-            p_custom = a
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Script which predicts threshold using specific model")
 
-    scenes = os.listdir(scenes_path)
+    parser.add_argument('--interval', type=str, help='Interval value to keep from svd', default='"0, 200"')
+    parser.add_argument('--model', type=str, help='.joblib or .json file (sklearn or keras model)')
+    parser.add_argument('--mode', type=str, help='Kind of normalization level wished', choices=normalization_choices)
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=metric_choices)
+    #parser.add_argument('--limit_detection', type=int, help='Specify number of same prediction to stop threshold prediction', default=2)
+    parser.add_argument('--custom', type=str, help='Name of custom min max file if use of renormalization of data', default=False)
+
+    args = parser.parse_args()
 
+    p_interval   = list(map(int, args.interval.split(',')))
+    p_model_file = args.model
+    p_mode       = args.mode
+    p_metric     = args.metric
+    #p_limit      = args.limit
+    p_custom     = args.custom
+
+    scenes = os.listdir(scenes_path)
     scenes = [s for s in scenes if s in maxwell_scenes]
 
     # go ahead each scenes

predict_seuil_expe_maxwell_curve.py

@@ -5,7 +5,7 @@ import numpy as np
 from ipfml import processing
 from PIL import Image
 
-import sys, os, getopt
+import sys, os, argparse
 import subprocess
 import time
 
@@ -21,6 +21,8 @@ threshold_map_file_prefix = cfg.threshold_map_folder + "_"
 
 zones                     = cfg.zones_indices
 maxwell_scenes            = cfg.maxwell_scenes_names
+normalization_choices     = cfg.normalization_choices
+metric_choices            = cfg.metric_choices_labels
 
 simulation_curves_zones   = "simulation_curves_zones_"
 tmp_filename              = '/tmp/__model__img_to_predict.png'
@@ -31,42 +33,26 @@ current_dirpath = os.getcwd()
 def main():
 
     p_custom = False
+        
+    parser = argparse.ArgumentParser(description="Script which predicts threshold using specific model")
 
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python predict_seuil_expe_maxwell_curve.py --interval "0,20" --model path/to/xxxx.joblib --mode svdn --metric lab --limit_detection xx --custom min_max_filename')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "ht:m:o:l:c", ["help=", "interval=", "model=", "mode=", "metric=", "limit_detection=", "custom="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python predict_seuil_expe_maxwell_curve.py --interval "xx,xx" --model path/to/xxxx.joblib --mode svdn --metric lab --limit_detection xx --custom min_max_filename')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python predict_seuil_expe_maxwell_curve.py --interval "xx,xx" --model path/to/xxxx.joblib --mode svdn --metric lab --limit_detection xx --custom min_max_filename')
-            sys.exit()
-        elif o in ("-t", "--interval"):
-            p_interval = a
-        elif o in ("-m", "--model"):
-            p_model_file = a
-        elif o in ("-o", "--mode"):
-            p_mode = a
-
-            if p_mode != 'svdn' and p_mode != 'svdne' and p_mode != 'svd':
-                assert False, "Mode not recognized"
-
-        elif o in ("-m", "--metric"):
-            p_metric = a
-        elif o in ("-l", "--limit_detection"):
-            p_limit = int(a)
-        elif o in ("-c", "--custom"):
-            p_custom = a
-        else:
-            assert False, "unhandled option"
+    parser.add_argument('--interval', type=str, help='Interval value to keep from svd', default='"0, 200"')
+    parser.add_argument('--model', type=str, help='.joblib or .json file (sklearn or keras model)')
+    parser.add_argument('--mode', type=str, help='Kind of normalization level wished', choices=normalization_choices)
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=metric_choices)
+    #parser.add_argument('--limit_detection', type=int, help='Specify number of same prediction to stop threshold prediction', default=2)
+    parser.add_argument('--custom', type=str, help='Name of custom min max file if use of renormalization of data', default=False)
 
-    scenes = os.listdir(scenes_path)
+    args = parser.parse_args()
+
+    p_interval   = list(map(int, args.interval.split(',')))
+    p_model_file = args.model
+    p_mode       = args.mode
+    p_metric     = args.metric
+    #p_limit      = args.limit
+    p_custom     = args.custom
 
+    scenes = os.listdir(scenes_path)
     scenes = [s for s in scenes if s in maxwell_scenes]
 
     print(scenes)

prediction_scene.py

@@ -11,7 +11,7 @@ from keras import backend as K
 from keras.models import model_from_json
 from keras.wrappers.scikit_learn import KerasClassifier
 
-import sys, os, getopt
+import sys, os, argparse
 import json
 
 from modules.utils import config as cfg
@@ -19,31 +19,20 @@ from modules.utils import config as cfg
 output_model_folder = cfg.saved_models_folder
 
 def main():
+    
+    parser = argparse.ArgumentParser(description="Give model performance on specific scene")
 
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python prediction_scene.py --data xxxx.csv --model xxxx.joblib --output xxxx --scene xxxx')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hd:o:s", ["help=", "data=", "model=", "output=", "scene="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python prediction_scene.py --data xxxx.csv --model xxxx.joblib --output xxxx --scene xxxx')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python prediction_scene.py --data xxxx.csv --model xxxx.joblib --output xxxx --scene xxxx')
-            sys.exit()
-        elif o in ("-d", "--data"):
-            p_data_file = a
-        elif o in ("-m", "--model"):
-            p_model_file = a
-        elif o in ("-o", "--output"):
-            p_output = a
-        elif o in ("-s", "--scene"):
-            p_scene = a
-        else:
-            assert False, "unhandled option"
+    parser.add_argument('--data', type=str, help='dataset filename prefix of specific scene (without .train and .test)')
+    parser.add_argument('--model', type=str, help='saved model (Keras or SKlearn) filename with extension')
+    parser.add_argument('--output', type=str, help="filename to store predicted and performance model obtained on scene")
+    parser.add_argument('--scene', type=str, help="scene indice to predict", choices=cfg.scenes_indices)
+
+    args = parser.parse_args()
+
+    p_data_file  = args.data
+    p_model_file = args.model
+    p_output     = args.output
+    p_scene      = args.scene
 
     if '.joblib' in p_model_file:
         kind_model = 'sklearn'

save_model_result_in_md.py

@@ -5,7 +5,7 @@ import numpy as np
 from ipfml import processing
 from PIL import Image
 
-import sys, os, getopt
+import sys, os, argparse
 import subprocess
 import time
 
@@ -22,33 +22,19 @@ current_dirpath = os.getcwd()
 
 def main():
 
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python save_model_result_in_md.py --interval "0,20" --model path/to/xxxx.joblib --mode ["svd", "svdn", "svdne"] --metric ["lab", "mscn"]')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "ht:m:o:l", ["help=", "interval=", "model=", "mode=", "metric="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python save_model_result_in_md.py --interval "xx,xx" --model path/to/xxxx.joblib --mode ["svd", "svdn", "svdne"] --metric ["lab", "mscn"]')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python save_model_result_in_md.py --interval "xx,xx" --model path/to/xxxx.joblib --mode ["svd", "svdn", "svdne"] --metric ["lab", "mscn"]')
-            sys.exit()
-        elif o in ("-t", "--interval"):
-            p_interval = list(map(int, a.split(',')))
-        elif o in ("-m", "--model"):
-            p_model_file = a
-        elif o in ("-o", "--mode"):
-            p_mode = a
-
-            if p_mode != 'svdn' and p_mode != 'svdne' and p_mode != 'svd':
-                assert False, "Mode not recognized"
-        elif o in ("-me", "--metric"):
-            p_metric = a
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Display SVD data of scene zone")
+
+    parser.add_argument('--interval', type=str, help='Interval value to keep from svd', default='"0, 200"')
+    parser.add_argument('--model', type=str, help='.joblib or .json file (sklearn or keras model)')
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=cfg.metric_choices)
+    parser.add_argument('--mode', type=str, help='Kind of normalization level wished', choices=cfg.normalization_choices)
+
+    args = parser.parse_args()
+    
+    p_interval   = list(map(int, args.interval.split(',')))
+    p_model_file = args.model
+    p_metric     = args.metric
+    p_mode       = args.mode
 
 
     # call model and get global result in scenes

save_model_result_in_md_maxwell.py

@@ -18,7 +18,7 @@ import pandas as pd
 from ipfml import processing
 from PIL import Image
 
-import sys, os, getopt
+import sys, os, argparse
 import subprocess
 import time
 import json
@@ -41,34 +41,21 @@ def main():
 
     kind_model = 'keras'
     model_ext = ''
+    
+    parser = argparse.ArgumentParser(description="Display SVD data of scene zone")
+
+    parser.add_argument('--interval', type=str, help='Interval value to keep from svd', default='"0, 200"')
+    parser.add_argument('--model', type=str, help='.joblib or .json file (sklearn or keras model)')
+    parser.add_argument('--metric', type=str, help='Metric data choice', choices=cfg.metric_choices)
+    parser.add_argument('--mode', type=str, help='Kind of normalization level wished', choices=cfg.normalization_choices)
+
+    args = parser.parse_args()
+
+    p_interval   = list(map(int, args.interval.split(',')))
+    p_model_file = args.model
+    p_metric     = args.metric
+    p_mode       = args.mode
 
-    if len(sys.argv) <= 1:
-        print('Run with default parameters...')
-        print('python save_model_result_in_md.py --interval "0,20" --model path/to/xxxx.joblib --mode ["svd", "svdn", "svdne"] --metric ["lab", "mscn"]')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "ht:m:o:l", ["help=", "interval=", "model=", "mode=", "metric="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python save_model_result_in_md.py --interval "xx,xx" --model path/to/xxxx.joblib --mode ["svd", "svdn", "svdne"] --metric ["lab", "mscn"]')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python save_model_result_in_md.py --interval "xx,xx" --model path/to/xxxx.joblib --mode ["svd", "svdn", "svdne"] --metric ["lab", "mscn"]')
-            sys.exit()
-        elif o in ("-t", "--interval"):
-            p_interval = list(map(int, a.split(',')))
-        elif o in ("-m", "--model"):
-            p_model_file = a
-        elif o in ("-o", "--mode"):
-            p_mode = a
-
-            if p_mode != 'svdn' and p_mode != 'svdne' and p_mode != 'svd':
-                assert False, "Mode not recognized"
-        elif o in ("-m", "--metric"):
-            p_metric = a
-        else:
-            assert False, "unhandled option"
 
     # call model and get global result in scenes
     begin, end = p_interval

train_model.py

@@ -11,7 +11,7 @@ from sklearn.model_selection import cross_val_score
 
 import numpy as np
 import pandas as pd
-import sys, os, getopt
+import sys, os, argparse
 
 from modules.utils import config as cfg
 from modules import models as mdl
@@ -25,31 +25,17 @@ output_model_folder = os.path.join(current_dirpath, saved_models_folder)
 
 def main():
 
-    if len(sys.argv) <= 2:
-        print('python train_model.py --data xxxx --output xxxx --choice svm_model')
-        sys.exit(2)
-    try:
-        opts, args = getopt.getopt(sys.argv[1:], "hd:o:c", ["help=", "data=", "output=", "choice="])
-    except getopt.GetoptError:
-        # print help information and exit:
-        print('python train_model.py --data xxxx --output xxxx --choice svm_model')
-        sys.exit(2)
-    for o, a in opts:
-        if o == "-h":
-            print('python train_model.py --data xxxx --output xxxx --choice svm_model')
-            sys.exit()
-        elif o in ("-d", "--data"):
-            p_data_file = a
-        elif o in ("-o", "--output"):
-            p_output = a
-        elif o in ("-c", "--choice"):
-            p_choice = a
-
-            if not p_choice in models_list:
-                assert False, "Unknown model choice"
-
-        else:
-            assert False, "unhandled option"
+    parser = argparse.ArgumentParser(description="Train SKLearn model and save it into .joblib file")
+
+    parser.add_argument('--data', type=str, help='dataset filename prefix (without .train and .test)')
+    parser.add_argument('--output', type=str, help='output file name desired for model (without .joblib extension)')
+    parser.add_argument('--choice', type=str, help='model choice from list of choices', choices=models_list)
+
+    args = parser.parse_args()
+
+    p_data_file = args.data
+    p_output    = args.output
+    p_choice    = args.choice
 
     if not os.path.exists(output_model_folder):
         os.makedirs(output_model_folder)
@@ -129,7 +115,6 @@ def main():
     val_f1 = f1_score(y_val, y_val_model)
     test_f1 = f1_score(y_test, y_test_model)
 
-
     ###################
     # 5. Output : Print and write all information in csv
     ###################
@@ -141,7 +126,6 @@ def main():
     print("Test: ", val_accuracy)
     print("Test F1: ", test_f1)
 
-
     ##################
     # 6. Save model : create path if not exists
     ##################