Study of synthesis images noise detection using 26 attributes

Jérôme BUISINE 21bd064e0f enable filter of attributes		преди 3 години
OpenML_datasets	f94ba1ea70 update openML datasets used	преди 4 години
analysis	a18c913871 Update train surrogate model	преди 3 години
features_selection	48c4349333 sorted open ml problems in order to well restart	преди 4 години
generate	c409191b1c update prediction script for new dataset structure	преди 4 години
modules @ cebf2adbf1	0a05939b74 Update of optimization process using backups	преди 5 години
optimization	75bfebb06b Remove add of surrogate model during local search	преди 3 години
prediction	21bd064e0f enable filter of attributes	преди 3 години
rnn @ 3e0abd40fe	c863576804 New rnn submodule version	преди 3 години
utils	2fc4db3bfb use of wsao module for accelerate ILS (using surrogate)	преди 4 години
wsao @ a92ca5a285	c41293b6fa computation of mae and save it	преди 4 години
.gitignore	475f2851d1 Add of ocurences displayed	преди 3 години
.gitmodules	41659bc1c2 add of training using rnn models	преди 4 години
LICENSE	dc0463b6b5 Project initialization	преди 5 години
README.md	30d257e0f7 update of the whole project to enable use of new dataset	преди 4 години
check_random_forest_perfomance.py	8dc3803465 update number of features as input	преди 3 години
check_random_forest_perfomance_rfe.py	bc9d7037b7 Enable merge from rfe script	преди 3 години
custom_config.py	e4f5839e36 Use surrogate from scract as proposed framework	преди 4 години
data_attributes.py	a4119a186e update kolmogorov attributes	преди 4 години
find_best_attributes.py	6032efa1b1 Use of population for rendering surrogate	преди 3 години
find_best_attributes_from.py	a2fb893050 svm can now be used for selector	преди 4 години
find_best_attributes_no_surrogate.py	4546ab0351 Lunch using no surrogate for comparisons	преди 3 години
find_best_attributes_surrogate.py	38ca51bff9 use of random forest	преди 3 години
find_best_attributes_surrogate_dl.py	6032efa1b1 Use of population for rendering surrogate	преди 3 години
find_best_attributes_surrogate_openML.py	6032efa1b1 Use of population for rendering surrogate	преди 3 години
find_best_attributes_surrogate_openML_multi.py	6032efa1b1 Use of population for rendering surrogate	преди 3 години
find_best_attributes_surrogate_openML_multi_specific.py	6032efa1b1 Use of population for rendering surrogate	преди 3 години
find_best_filters.py	6032efa1b1 Use of population for rendering surrogate	преди 3 години
models.py	0b94c18199 update number of parameters for svm	преди 3 години
requirements.txt	c52c6fae6c now use of macop Python package for optimization process	преди 4 години
run_no_surrogate_rendering.sh	1e890c2840 Update bash script	преди 3 години
run_openML_surrogate.py	48c4349333 sorted open ml problems in order to well restart	преди 4 години
run_openML_surrogate_multi.py	c41293b6fa computation of mae and save it	преди 4 години
run_openML_surrogate_multi_specific.py	33cf98b131 enable run of commands	преди 4 години
run_surrogate_rendering.sh	b111f51621 Reduce pop size	преди 3 години
train_model.py	c409191b1c update prediction script for new dataset structure	преди 4 години
train_model_attributes.py	bc9d7037b7 Enable merge from rfe script	преди 3 години
train_model_filters.py	b73b27ab44 add balanced data into SVC	преди 4 години

Noise detection using 26 attributes

Description

Noise detection on synthesis images with 26 attributes obtained using few filters.

Filters list:

average
wiener
median
gaussian
wavelet

Requirements

pip install -r requirements.txt

Project structure

Link to your dataset

You need database which respects this structure:

dataset/
- Scene1/
- Scene1_00050.png
- Scene1_00070.png
- ...
- Scene1_01180.png
- Scene1_01200.png
- Scene2/
- ...
- ...

Code architecture description

modules/*: contains all modules usefull for the whole project (such as configuration variables)
analysis/*: contains all jupyter notebook used for analysis during thesis
generate/*: contains python scripts for generate data from scenes (described later)
data_processing/*: all python scripts for generate custom dataset for models
prediction/*: all python scripts for predict new threshold from computed models
data_attributes.py: files which contains all extracted features implementation from an image.
custom_config.py: override the main configuration project of modules/config/global_config.py
train_model.py: script which is used to run specific model available.

Generated data directories:

data/*: folder which will contain all generated .train & .test files in order to train model.
data/saved_models/*: all scikit learn or keras models saved.
data/models_info/*: all markdown files generated to get quick information about model performance and prediction obtained after running run/runAll_*.sh script.
data/results/: This folder contains model_comparisons.csv file used for store models performance.

License

The MIT license

README.md