[otb-users] No-data in Random Forest Classifier

['O Buck' via otb-users]

Dear all,

does anybody know, how the implemented RF (Random Forest) classifier treats
no data values. I have a layerstack image consisting of n-Bands. Bands 1-6
represent one image acqisition date t1 (blue, green, red....), Bands 7-12
represent the image date t2...,

Within each band there are no-data values (set to -1) caused by a
preprocessing step masking clouds, shadows, image artefacts ....


If I put this layerstack into the RF classifier, there classification
results is produced, yet I am not sure about the no-data areas. How are
they treated by the RF. Is it tolerable to include the no-data values. Are
there better classifiers implemented?

Regards
Oliver
--
--
Check the OTB FAQ at
http://www.orfeo-toolbox.org/FAQ.html

You received this message because you are subscribed to the Google
Groups "otb-users" group.
To post to this group, send email to otb-***@googlegroups.com
To unsubscribe from this group, send email to
otb-users+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/otb-users?hl=en
---
You received this message because you are subscribed to the Google Groups "otb-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to otb-users+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Cédric Traizet]

Hi Oliver,

A no-data value is treated like any other value by the classifier and
used for the classification.

If you don't want to process pixels with no-data value(s),  you could
build a no-data mask from your input image and the ManageNoData
application in buildmask mode
(https://www.orfeo-toolbox.org/CookBook/Applications/app_ManageNoData.html)
and use it as mask parameter of the ImageClassifier application (masked
pixels will have the class label 0).

Sincerely,

Cédric

Post by 'O Buck' via otb-users
Dear all,
does anybody know, how the implemented RF (Random Forest) classifier
treats no data values. I have a layerstack image consisting of
n-Bands. Bands 1-6 represent one image acqisition date t1 (blue,
green, red....), Bands 7-12 represent the image date t2...,
Within each band there are no-data values (set to -1) caused by a
preprocessing step masking clouds, shadows, image artefacts ....
If I put this layerstack into the RF classifier, there classification
results is produced, yet I am not sure about the no-data areas. How
are they treated by the RF. Is it tolerable to include the no-data
values. Are there better classifiers implemented?
Regards
Oliver
--
--
Check the OTB FAQ at
http://www.orfeo-toolbox.org/FAQ.html
You received this message because you are subscribed to the Google
Groups "otb-users" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/otb-users?hl=en
---
You received this message because you are subscribed to the Google
Groups "otb-users" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--
--
Check the OTB FAQ at
http://www.orfeo-toolbox.org/FAQ.html

You received this message because you are subscribed to the Google
Groups "otb-users" group.
To post to this group, send email to otb-***@googlegroups.com
To unsubscribe from this group, send email to
otb-users+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/otb-users?hl=en
---
You received this message because you are subscribed to the Google Groups "otb-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to otb-users+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[lewirbi]

Hello Jordi
I have a problem using the imagetimeseries gapfilling application. i have
concatenated the 20m resolution bands and have also prepared the masks for
the different bands but now do not know exactly which mask to use, or do i
also need to concatenate the masks following the same sequence of bands and
dates like the input time series image (concatenated)?

Hy Jordi,
thanks Jordi. The problem is, that my time stack is not very dense
(Landsat from 1990). I have a maximum of 10 images per path/row, if I allow
max cloud cover of 80%.
Have you had experience with Landsat-5 data and the TimeSeriesGapFilling
tool?
My focus classes are agriculture, so I can not interpolate the data too
far.
Regards
Oliver

Hi,
If you have time series, you can use the ImageTimeSeriesGapFilling
application which will interpolate the time profile if you provide the
corresponding cloud masks.
Jordi
On Fri 08-Jun-2018 at 15:42:31 +0200, "'O Buck' via otb-users" <

Thanks Cedric for your reply.
Yeap I was thinking of this. Problem is that I have pretty cloudy

images (old Landsat 5 data from 1990), which means I have "no-data" cloud
pixels in all bands at different locations. If I mask out all pixels that
have a no-data value in at least one band, my layerstack image will hardly

contain any "valid" pixels.
I was hoping that the RF would ignore no-data values and train with

whatever pixel dimension it has. Lets say I have a multi-temporal
layerstack image conposed of five aquisition images with 6 bands each (i.e.
30 bands in total), the pixel information available would range from 0
(always

cloudy) to 30 bands (never clouds).
But maybe I am completely wrong, due to my lack of deeper RF knowledge.
Oliver
Hi Oliver,
A no-data value is treated like any other value by the classifier and

used for the classification.

If you don't want to process pixels with no-data value(s), you could

build a no-data mask from your input image and the ManageNoData application
in buildmask mode (
https://www.orfeo-toolbox.org/CookBook/Applications/app_ManageNoData.html)
and use it as mask parameter

of the ImageClassifier application (masked pixels will have the class

label 0).

Sincerely,
Cédric
Dear all,
does anybody know, how the implemented RF (Random Forest) classifier

treats no data values. I have a layerstack image consisting of n-Bands.
Bands 1-6 represent one image acqisition date t1 (blue, green, red....),
Bands 7-12 represent the image date t2...,

Within each band there are no-data values (set to -1) caused by a

preprocessing step masking clouds, shadows, image artefacts ....

If I put this layerstack into the RF classifier, there classification

results is produced, yet I am not sure about the no-data areas. How are
they treated by the RF. Is it tolerable to include the no-data values. Are
there better classifiers implemented?

Regards
Oliver
--
--
Check the OTB FAQ at
http://www.orfeo-toolbox.org/FAQ.html
You received this message because you are subscribed to the Google
Groups "otb-users" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/otb-users?hl=en
---
You received this message because you are subscribed to the Google

Groups "otb-users" group.

To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.
--

--
--
Check the OTB FAQ at
http://www.orfeo-toolbox.org/FAQ.html
You received this message because you are subscribed to the Google
Groups "otb-users" group.
<javascript:>
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/otb-users?hl=en
---
You received this message because you are subscribed to a topic in the
Google Groups "otb-users" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/otb-users/aanhuGRcxos/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
For more options, visit https://groups.google.com/d/optout.

--
--
Check the OTB FAQ at
http://www.orfeo-toolbox.org/FAQ.html

You received this message because you are subscribed to the Google
Groups "otb-users" group.
To post to this group, send email to otb-***@googlegroups.com
To unsubscribe from this group, send email to
otb-users+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/otb-users?hl=en
---
You received this message because you are subscribed to the Google Groups "otb-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to otb-users+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jordi Inglada]

Post by lewirbi
Hello Jordi
I have a problem using the imagetimeseries gapfilling application. i have concatenated the 20m resolution bands and have also prepared the masks for the different bands but now do not know exactly which mask to use, or do i also need to concatenate the masks following the same
sequence of bands and dates like the input time series image (concatenated)?

Hi,

The masks have to be concatenated following the same order as the images. However, only one mask channel per date is neede indenpendently of the number of bands of each image date. There is a parameter "-comp" which allows to set the number of components (bands) per date so that everything is coherent.

Jordi

Post by lewirbi
Hy Jordi,
thanks Jordi. The problem is, that my time stack is not very dense (Landsat from 1990). I have a maximum of 10 images per path/row, if I allow max cloud cover of 80%.
Have you had experience with Landsat-5 data and the TimeSeriesGapFilling tool?
My focus classes are agriculture, so I can not interpolate the data too far.
Regards
Oliver
Hi,
If you have time series, you can use the ImageTimeSeriesGapFilling application which will interpolate the time profile if you provide the corresponding cloud masks.
Jordi

Thanks Cedric for your reply.
Yeap I was thinking of this. Problem is that I have pretty cloudy images (old Landsat 5 data from 1990), which means I have "no-data" cloud pixels in all bands at different locations. If I mask out all pixels that have a no-data value in at least one band, my layerstack

image will hardly

contain any "valid" pixels.
I was hoping that the RF would ignore no-data values and train with whatever pixel dimension it has. Lets say I have a multi-temporal layerstack image conposed of five aquisition images with 6 bands each (i.e. 30 bands in total), the pixel information available would range

from 0 (always

cloudy) to 30 bands (never clouds).
But maybe I am completely wrong, due to my lack of deeper RF knowledge.
Oliver
Hi Oliver,
A no-data value is treated like any other value by the classifier and used for the classification.
If you don't want to process pixels with no-data value(s), you could build a no-data mask from your input image and the ManageNoData application in buildmask mode (https://www.orfeo-toolbox.org/CookBook/Applications/app_ManageNoData.html) and use it as mask

parameter

of the ImageClassifier application (masked pixels will have the class label 0).
Sincerely,
Cédric
Dear all,
does anybody know, how the implemented RF (Random Forest) classifier treats no data values. I have a layerstack image consisting of n-Bands. Bands 1-6 represent one image acqisition date t1 (blue, green, red....), Bands 7-12 represent the image date t2...,
Within each band there are no-data values (set to -1) caused by a preprocessing step masking clouds, shadows, image artefacts ....
If I put this layerstack into the RF classifier, there classification results is produced, yet I am not sure about the no-data areas. How are they treated by the RF. Is it tolerable to include the no-data values. Are there better classifiers implemented?
Regards
Oliver
--
--
Check the OTB FAQ at
http://www.orfeo-toolbox.org/FAQ.html
You received this message because you are subscribed to the Google
Groups "otb-users" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/otb-users?hl=en
---
You received this message because you are subscribed to the Google Groups "otb-users" group.
For more options, visit https://groups.google.com/d/optout.
--

--
--
Check the OTB FAQ at
http://www.orfeo-toolbox.org/FAQ.html
You received this message because you are subscribed to the Google
Groups "otb-users" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/otb-users?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "otb-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/otb-users/aanhuGRcxos/unsubscribe.
For more options, visit https://groups.google.com/d/optout.
--

--
--
Check the OTB FAQ at
http://www.orfeo-toolbox.org/FAQ.html

You received this message because you are subscribed to the Google
Groups "otb-users" group.
To post to this group, send email to otb-***@googlegroups.com
To unsubscribe from this group, send email to
otb-users+***@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/otb-users?hl=en
---
You received this message because you are subscribed to the Google Groups "otb-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to otb-users+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.