torchgeo.models

Change Star

class torchgeo.models.ChangeStar(dense_feature_extractor, seg_classifier, changemixin, inference_mode='t1t2')

Bases: Module

The base class of the network architecture of ChangeStar.

ChangeStar is composed of an any segmentation model and a ChangeMixin module. This model is mainly used for binary/multi-class change detection under bitemporal supervision and single-temporal supervision. It features the property of segmentation architecture reusing, which is helpful to integrate advanced dense prediction (e.g., semantic segmentation) network architecture into change detection.

For multi-class change detection, semantic change prediction can be inferred by a binary change prediction from the ChangeMixin module and two semantic predictions from the Segmentation model.

If you use this model in your research, please cite the following paper:

• https://arxiv.org/abs/2108.07002

__init__(dense_feature_extractor, seg_classifier, changemixin, inference_mode='t1t2')

Initializes a new ChangeStar model.

Parameters:

• dense_feature_extractor (Module) – module for dense feature extraction, typically a semantic segmentation model without semantic segmentation head.

• seg_classifier (Module) – semantic segmentation head, typically a convolutional layer followed by an upsampling layer.

• changemixin (ChangeMixin) – torchgeo.models.ChangeMixin module

• inference_mode (str) – name of inference mode 't1t2' | 't2t1' | 'mean'. 't1t2': concatenate bitemporal features in the order of t1->t2; 't2t1': concatenate bitemporal features in the order of t2->t1; 'mean': the weighted mean of the output of 't1t2' and 't1t2'

forward(x)

Forward pass of the model.

Parameters:

x (Tensor) – a bitemporal input tensor of shape [B, T, C, H, W]

Returns:

a dictionary containing bitemporal semantic segmentation logit and binary change detection logit/probability

Return type:

Dict[str, Tensor]

class torchgeo.models.ChangeStarFarSeg(backbone='resnet50', classes=1, backbone_pretrained=True)

Bases: ChangeStar

The network architecture of ChangeStar(FarSeg).

ChangeStar(FarSeg) is composed of a FarSeg model and a ChangeMixin module.

If you use this model in your research, please cite the following paper:

• https://arxiv.org/abs/2108.07002

__init__(backbone='resnet50', classes=1, backbone_pretrained=True)

Initializes a new ChangeStarFarSeg model.

Parameters:

• backbone (str) – name of ResNet backbone

• classes (int) – number of output segmentation classes

• backbone_pretrained (bool) – whether to use pretrained weight for backbone

class torchgeo.models.ChangeMixin(in_channels=256, inner_channels=16, num_convs=4, scale_factor=4.0)

Bases: Module

This module enables any segmentation model to detect binary change.

The common usage is to attach this module on a segmentation model without the classification head.

If you use this model in your research, please cite the following paper:

• https://arxiv.org/abs/2108.07002

__init__(in_channels=256, inner_channels=16, num_convs=4, scale_factor=4.0)

Initializes a new ChangeMixin module.

Parameters:

• in_channels (int) – sum of channels of bitemporal feature maps

• inner_channels (int) – number of channels of inner feature maps

• num_convs (int) – number of convolution blocks

• scale_factor (float) – number of upsampling factor

forward(bi_feature)

Forward pass of the model.

Parameters:

bi_feature (Tensor) – input bitemporal feature maps of shape [b, t, c, h, w]

Returns:

a list of bidirected output predictions

Return type:

List[Tensor]

FarSeg

class torchgeo.models.FarSeg(backbone='resnet50', classes=16, backbone_pretrained=True)

Bases: Module

Foreground-Aware Relation Network (FarSeg).

This model can be used for binary- or multi-class object segmentation, such as building, road, ship, and airplane segmentation. It can be also extended as a change detection model. It features a foreground-scene relation module to model the relation between scene embedding, object context, and object feature, thus improving the discrimination of object feature representation.

If you use this model in your research, please cite the following paper:

• https://arxiv.org/pdf/2011.09766.pdf

__init__(backbone='resnet50', classes=16, backbone_pretrained=True)

Initialize a new FarSeg model.

Parameters:

• backbone (str) – name of ResNet backbone, one of [“resnet18”, “resnet34”, “resnet50”, “resnet101”]

• classes (int) – number of output segmentation classes

• backbone_pretrained (bool) – whether to use pretrained weight for backbone

forward(x)

Forward pass of the model.

Parameters:

x (Tensor) – input image

Returns:

output prediction

Return type:

Tensor

Fully-convolutional Network

class torchgeo.models.FCN(in_channels, classes, num_filters=64)

Bases: Module

A simple 5 layer FCN with leaky relus and ‘same’ padding.

__init__(in_channels, classes, num_filters=64)

Initializes the 5 layer FCN model.

Parameters:

• in_channels (int) – Number of input channels that the model will expect

• classes (int) – Number of filters in the final layer

• num_filters (int) – Number of filters in each convolutional layer

forward(x)

Forward pass of the model.

FC Siamese Networks

class torchgeo.models.FCSiamConc(encoder_name='resnet34', encoder_depth=5, encoder_weights='imagenet', decoder_use_batchnorm=True, decoder_channels=(256, 128, 64, 32, 16), decoder_attention_type=None, in_channels=3, classes=1, activation=None)

Bases: SegmentationModel

Fully-convolutional Siamese Concatenation (FC-Siam-conc).

If you use this model in your research, please cite the following paper:

• https://doi.org/10.1109/ICIP.2018.8451652

__init__(encoder_name='resnet34', encoder_depth=5, encoder_weights='imagenet', decoder_use_batchnorm=True, decoder_channels=(256, 128, 64, 32, 16), decoder_attention_type=None, in_channels=3, classes=1, activation=None)

Initialize a new FCSiamConc model.

Parameters:

• encoder_name (str) – Name of the classification model that will be used as an encoder (a.k.a backbone) to extract features of different spatial resolution

• encoder_depth (int) – A number of stages used in encoder in range [3, 5]. two times smaller in spatial dimensions than previous one (e.g. for depth 0 we will have features. Each stage generate features with shapes [(N, C, H, W),], for depth 1 - [(N, C, H, W), (N, C, H // 2, W // 2)] and so on). Default is 5

• encoder_weights (Optional[str]) – One of None (random initialization), “imagenet” (pre-training on ImageNet) and other pretrained weights (see table with available weights for each encoder_name)

• decoder_channels (Sequence[int]) – List of integers which specify in_channels parameter for convolutions used in decoder. Length of the list should be the same as encoder_depth

• decoder_use_batchnorm (bool) – If True, BatchNorm2d layer between Conv2D and Activation layers is used. If “inplace” InplaceABN will be used, allows to decrease memory consumption. Available options are True, False, “inplace”

• decoder_attention_type (Optional[str]) – Attention module used in decoder of the model. Available options are None and scse. SCSE paper https://arxiv.org/abs/1808.08127

• in_channels (int) – A number of input channels for the model, default is 3 (RGB images)

• classes (int) – A number of classes for output mask (or you can think as a number of channels of output mask)

• activation (Optional[Union[str, Callable[[Tensor], Tensor]]]) – An activation function to apply after the final convolution n layer. Available options are “sigmoid”, “softmax”, “logsoftmax”, “tanh”, “identity”, callable and None. Default is None

forward(x)

Forward pass of the model.

Parameters:

x (Tensor) – input images of shape (b, t, c, h, w)

Returns:

predicted change masks of size (b, classes, h, w)

Return type:

Tensor

class torchgeo.models.FCSiamDiff(*args, **kwargs)

Bases: Unet

Fully-convolutional Siamese Difference (FC-Siam-diff).

If you use this model in your research, please cite the following paper:

• https://doi.org/10.1109/ICIP.2018.8451652

__init__(*args, **kwargs)

Initialize a new FCSiamConc model.

Parameters:

• encoder_name – Name of the classification model that will be used as an encoder (a.k.a backbone) to extract features of different spatial resolution

• encoder_depth – A number of stages used in encoder in range [3, 5]. two times smaller in spatial dimensions than previous one (e.g. for depth 0 we will have features. Each stage generate features with shapes [(N, C, H, W),], for depth 1 - [(N, C, H, W), (N, C, H // 2, W // 2)] and so on). Default is 5

• encoder_weights – One of None (random initialization), “imagenet” (pre-training on ImageNet) and other pretrained weights (see table with available weights for each encoder_name)

• decoder_channels – List of integers which specify in_channels parameter for convolutions used in decoder. Length of the list should be the same as encoder_depth

• decoder_use_batchnorm – If True, BatchNorm2d layer between Conv2D and Activation layers is used. If “inplace” InplaceABN will be used, allows to decrease memory consumption. Available options are True, False, “inplace”

• decoder_attention_type – Attention module used in decoder of the model. Available options are None and scse. SCSE paper https://arxiv.org/abs/1808.08127

• in_channels – A number of input channels for the model, default is 3 (RGB images)

• classes – A number of classes for output mask (or you can think as a number of channels of output mask)

• activation – An activation function to apply after the final convolution n layer. Available options are “sigmoid”, “softmax”, “logsoftmax”, “tanh”, “identity”, callable and None. Default is None

forward(x)

Forward pass of the model.

Parameters:

x (Tensor) – input images of shape (b, t, c, h, w)

Returns:

predicted change masks of size (b, classes, h, w)

Return type:

Tensor

RCF Extractor

class torchgeo.models.RCF(in_channels=4, features=16, kernel_size=3, bias=-1.0, seed=None)

Bases: Module

This model extracts random convolutional features (RCFs) from its input.

RCFs are used in Multi-task Observation using Satellite Imagery & Kitchen Sinks (MOSAIKS) method proposed in https://www.nature.com/articles/s41467-021-24638-z.

Note

This Module is not trainable. It is only used as a feature extractor.

__init__(in_channels=4, features=16, kernel_size=3, bias=-1.0, seed=None)

Initializes the RCF model.

This is a static model that serves to extract fixed length feature vectors from input patches.

New in version 0.2: The seed parameter.

Parameters:

• in_channels (int) – number of input channels

• features (int) – number of features to compute, must be divisible by 2

• kernel_size (int) – size of the kernel used to compute the RCFs

• bias (float) – bias of the convolutional layer

• seed (Optional[int]) – random seed used to initialize the convolutional layer

forward(x)

Forward pass of the RCF model.

Parameters:

x (Tensor) – a tensor with shape (B, C, H, W)

Returns:

a tensor of size (B, self.num_features)

Return type:

Tensor

ResNet

torchgeo.models.resnet18(weights=None, *args, **kwargs)

ResNet-18 model.

If you use this model in your research, please cite the following paper:

• https://arxiv.org/pdf/1512.03385.pdf

New in version 0.4.

Parameters:

• weights (Optional[ResNet18_Weights]) – Pre-trained model weights to use.

• *args (Any) – Additional arguments to pass to timm.create_model()

• **kwargs (Any) – Additional keywork arguments to pass to timm.create_model()

Returns:

A ResNet-18 model.

Return type:

ResNet

torchgeo.models.resnet50(weights=None, *args, **kwargs)

ResNet-50 model.

If you use this model in your research, please cite the following paper:

• https://arxiv.org/pdf/1512.03385.pdf

Changed in version 0.4: Switched to multi-weight support API.

Parameters:

• weights (Optional[ResNet50_Weights]) – Pre-trained model weights to use.

• *args (Any) – Additional arguments to pass to timm.create_model().

• **kwargs (Any) – Additional keywork arguments to pass to timm.create_model().

Returns:

A ResNet-50 model.

Return type:

ResNet

class torchgeo.models.ResNet18_Weights(value)

Bases: WeightsEnum

ResNet18 weights.

For timm resnet18 implementation.

New in version 0.4.

class torchgeo.models.ResNet50_Weights(value)

Bases: WeightsEnum

ResNet50 weights.

For timm resnet50 implementation.

New in version 0.4.

Weight	Channels	Source	Citation	BigEarthNet	EuroSAT	So2Sat	OSCD
ResNet18_Weights.SENTINEL2_ALL_MOCO	13	link	link
ResNet18_Weights.SENTINEL2_RGB_MOCO	3	link	link
ResNet18_Weights.SENTINEL2_RGB_SECO	3	link	link	87.27	93.14		46.94
ResNet50_Weights.SENTINEL1_ALL_MOCO	2	link	link
ResNet50_Weights.SENTINEL2_ALL_DINO	13	link	link	90.7	99.1	63.6
ResNet50_Weights.SENTINEL2_ALL_MOCO	13	link	link	91.8	99.1	60.9
ResNet50_Weights.SENTINEL2_RGB_MOCO	3	link	link
ResNet50_Weights.SENTINEL2_RGB_SECO	3	link	link	87.81

Vision Transformer

torchgeo.models.vit_small_patch16_224(weights=None, *args, **kwargs)

Vision Transform (ViT) small patch size 16 model.

If you use this model in your research, please cite the following paper:

• https://arxiv.org/abs/2010.11929

New in version 0.4.

Parameters:

• weights (Optional[ViTSmall16_Weights]) – Pre-trained model weights to use.

• *args (Any) – Additional arguments to pass to timm.create_model().

• **kwargs (Any) – Additional keywork arguments to pass to timm.create_model().

Returns:

A ViT small 16 model.

Return type:

VisionTransformer

class torchgeo.models.ViTSmall16_Weights(value)

Bases: WeightsEnum

Vision Transformer Samll Patch Size 16 weights.

For timm vit_small_patch16_224 implementation.

New in version 0.4.

Weight	Channels	Source	Citation	BigEarthNet	EuroSAT	So2Sat	OSCD
ViTSmall16_Weights.SENTINEL2_ALL_DINO	13	link	link	90.5	99.0	62.2
ViTSmall16_Weights.SENTINEL2_ALL_MOCO	13	link	link	89.9	98.6	61.6

Utility Functions

torchgeo.models.get_model(name, *args, **kwargs)

Get an instantiated model from its name.

New in version 0.4.

Parameters:

• name (str) – Name of the model.

• *args (Any) – Additional arguments passed to the model builder method.

• **kwargs (Any) – Additional keyword arguments passed to the model builder method.

Returns:

An instantiated model.

Return type:

Module

torchgeo.models.get_model_weights(name)

Get the weights enum class associated with a given model.

New in version 0.4.

Parameters:

name (Union[Callable[[...], Module], str]) – Model builder function or the name under which it is registered.

Returns:

The weights enum class associated with the model.

Return type:

WeightsEnum

torchgeo.models.get_weight(name)

Get the weights enum value by its full name.

New in version 0.4.

Parameters:

name (str) – Name of the weight enum entry.

Returns:

The requested weight enum.

Return type:

WeightsEnum

torchgeo.models.list_models()

List the registered models.

New in version 0.4.

Returns:

A list of registered models.

Return type:

List[str]