Exploiting Objective Text Description of Images
for Visual Sentiment Analysis
Download the evaluation code
Download the code
(2017)
Objective Text Extraction
The folder 'cnn_processing' contains the scripts used to extract and store the Objective Text by exploiting the CNN models, as described in the paper. All the models have been pre-trained using Caffe, and are available at the following link:
The image captions have been extracted by using the NeuralTalk2 implementation provided at the following link:
Objective Feature Extraction
The folder 'textprocessing' contains the scripts used to perform the Objective Text pre-processing and the Objective Feature extraction (i.e., the extraction of the features OS, OT and OR).
Canonical Correlation Analysis
As in [1], the CCA embeddings have been obtained by exploiting the multi-view CCA implementation available at the following link:
Performance Evaluation
To run the code, execute the following command:
python test.py <Experiment ID>[_trunc] [C value]
for instance:python test.py O1
python test.py O1_trunc
python test.py O1_trunc 0.001
In the first case, the code performs a 10 fold cross-validation to find the best parameter C, with the feature extracted from the embedding space built according to the settings with Experiment ID O1 (i.e., with the features V T and OS).
In the second case, the truncated version of the Experiment ID O1 features will be used.
In the third case, the code will not perform the cross-validation, as the C parameter is given. Then, the code will perform the training and the test on the features corresponding to the specified Experiment ID, using the provided C parameter for SVM.
Each Experiment ID corresponds to a .mat input file.
To quickly try the code, download an example input file at the following link:
and execute the command:
python test.py O4_trunc
The entire repository containing all the input .mat files is accessible at the following links:
Back to Home
[1] Katsurai, Marie, and Shin'ichi Satoh. "Image sentiment analysis using latent correlations among visual, textual, and sentiment views." IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2016.