TensorRT/PluginFAQ

Why we need clone for Plugin interface?
TensorRT API documentation provides an explanation about this. Simply, it intends to share immutable resources, like input/output dimensions, workspace ptr or weights and etc, among different execution contexts and these immutable resources or parameters are created per each engine, and copied over the invocation of clone. NOTE: the digit ‘0x12458e00’ is the plugin object ptr and can be used to track when it gets destroyed. parsing: CropAndResizePluginCreator::createPlugin --> 0x12458e00 CropAndResizePlugin::CropAndResizePlugin CropAndResizePlugin::setPluginNamespace CropAndResizePlugin::getNbOutputs CropAndResizePlugin::clone --> 0x12459300 CropAndResizePlugin::CropAndResizePlugin CropAndResizePlugin::getNbOutputs CropAndResizePlugin::getOutputDataType CropAndResizePlugin::isOutputBroadcastAcrossBatch CropAndResizePlugin::getOutputDimensions parsing done
 * The following shows the callback flow of all IPlugin key APIs from network parsing, to engine building to inferencing (take reference from TensorRT 5.1.5).

engine building:   --> graph optimization, tactic and format selection, memory optimization and etc.		CropAndResizePlugin::clone --> 0xf9e8e0 CropAndResizePlugin::CropAndResizePlugin CropAndResizePlugin::supportsFormat CropAndResizePlugin::getOutputDataType

CropAndResizePlugin::clone --> 0x6380f750 CropAndResizePlugin::CropAndResizePlugin CropAndResizePlugin::supportsFormat CropAndResizePlugin::configurePlugin CropAndResizePlugin::getWorkspaceSize CropAndResizePlugin::initialize CropAndResizePlugin::getSerializationSize CropAndResizePlugin::serialize CropAndResizePlugin::destroy --> 0xf9e8e0 CropAndResizePlugin::~CropAndResizePlugin CropAndResizePlugin::getSerializationSize CropAndResizePlugin::serialize CropAndResizePlugin::destroy --> 0x12458e00 CropAndResizePlugin::~CropAndResizePlugin CropAndResizePlugin::destroy --> 0x12459300 CropAndResizePlugin::~CropAndResizePlugin engine building done

createExecutionContext and inference: CropAndResizePlugin::clone --> 0x166b2840 CropAndResizePlugin::CropAndResizePlugin CropAndResizePlugin::attachToContext CropAndResizePlugin::enqueue ropAndResizePlugin::detachFromContext CropAndResizePlugin::terminate CropAndResizePlugin::destroy --> 0x166b2840 CropAndResizePlugin::~CropAndResizePlugin CropAndResizePlugin::terminate CropAndResizePlugin::destroy --> 0x6380f750 CropAndResizePlugin::~CropAndResizePlugin Finished Deserializing: CropAndResizePluginCreator::deserializePlugin --> 0xd10c5b0 CropAndResizePlugin::CropAndResizePlugin CropAndResizePlugin::initialize Deserializing done createExecutionContext and infer: CropAndResizePlugin::enqueue CropAndResizePlugin::terminate CropAndResizePlugin::destroy CropAndResizePlugin::~CropAndResizePlugin Finished NOTE: TensorRT 5.1 has a known issue that initialize and terminate don’t appear in pairs (terminate will be invoked twice by different objects). If you want to allocate immutable resource for your plugin in initialize, you better make it as std::shared_ptr so that it won’t get released twice from terminate.
 * The following shows the callback flow from engine/plan deserializing to inferencing (take reference from TensorRT 5.1.5).