Difference between revisions of "TensorRT/PluginFAQ"
< TensorRT
m (→Why we need clone() for Plugin interface?) |
m (→Why we need clone() for Plugin interface?) |
||
(One intermediate revision by the same user not shown) | |||
Line 16: | Line 16: | ||
CropAndResizePlugin::isOutputBroadcastAcrossBatch() | CropAndResizePlugin::isOutputBroadcastAcrossBatch() | ||
CropAndResizePlugin::getOutputDimensions() | CropAndResizePlugin::getOutputDimensions() | ||
− | parsing done | + | parsing done |
− | engine building: --> graph optimization, tactic selection and | + | engine building: --> graph optimization, tactic and format selection, memory optimization and etc. |
CropAndResizePlugin::clone() --> 0xf9e8e0 | CropAndResizePlugin::clone() --> 0xf9e8e0 | ||
CropAndResizePlugin::CropAndResizePlugin() | CropAndResizePlugin::CropAndResizePlugin() | ||
Line 45: | Line 45: | ||
CropAndResizePlugin::destroy() --> 0x12459300 | CropAndResizePlugin::destroy() --> 0x12459300 | ||
CropAndResizePlugin::~CropAndResizePlugin() | CropAndResizePlugin::~CropAndResizePlugin() | ||
− | engine building done | + | engine building done |
− | createExecutionContext and | + | createExecutionContext and inference: |
CropAndResizePlugin::clone() --> 0x166b2840 | CropAndResizePlugin::clone() --> 0x166b2840 | ||
CropAndResizePlugin::CropAndResizePlugin() | CropAndResizePlugin::CropAndResizePlugin() | ||
Line 61: | Line 61: | ||
CropAndResizePlugin::destroy() --> 0x6380f750 | CropAndResizePlugin::destroy() --> 0x6380f750 | ||
CropAndResizePlugin::~CropAndResizePlugin() | CropAndResizePlugin::~CropAndResizePlugin() | ||
− | Finished | + | Finished |
</pre> | </pre> | ||
* The following shows the callback flow from engine/plan deserializing to inferencing (take reference from TensorRT 5.1.5). | * The following shows the callback flow from engine/plan deserializing to inferencing (take reference from TensorRT 5.1.5). | ||
<pre> | <pre> | ||
− | Deserializing: | + | Deserializing: |
CropAndResizePluginCreator::deserializePlugin --> 0xd10c5b0 | CropAndResizePluginCreator::deserializePlugin --> 0xd10c5b0 | ||
CropAndResizePlugin::CropAndResizePlugin | CropAndResizePlugin::CropAndResizePlugin |
Latest revision as of 23:35, 20 August 2019
Why we need clone() for Plugin interface?
TensorRT API documentation provides an explanation about this. Simply, it intends to share immutable resources, like input/output dimensions, workspace ptr or weights and etc, among different execution contexts and these immutable resources or parameters are created per each engine, and copied over the invocation of clone().
- The following shows the callback flow of all IPlugin key APIs from network parsing, to engine building to inferencing (take reference from TensorRT 5.1.5).
NOTE: the digit ‘0x12458e00’ is the plugin object ptr and can be used to track when it gets destroyed.
parsing: CropAndResizePluginCreator::createPlugin() --> 0x12458e00 CropAndResizePlugin::CropAndResizePlugin() CropAndResizePlugin::setPluginNamespace CropAndResizePlugin::getNbOutputs CropAndResizePlugin::clone() --> 0x12459300 CropAndResizePlugin::CropAndResizePlugin() CropAndResizePlugin::getNbOutputs() CropAndResizePlugin::getOutputDataType() CropAndResizePlugin::isOutputBroadcastAcrossBatch() CropAndResizePlugin::getOutputDimensions() parsing done engine building: --> graph optimization, tactic and format selection, memory optimization and etc. CropAndResizePlugin::clone() --> 0xf9e8e0 CropAndResizePlugin::CropAndResizePlugin() CropAndResizePlugin::supportsFormat() CropAndResizePlugin::getOutputDataType() CropAndResizePlugin::clone() --> 0x6380f750 CropAndResizePlugin::CropAndResizePlugin() CropAndResizePlugin::supportsFormat() CropAndResizePlugin::configurePlugin() CropAndResizePlugin::getWorkspaceSize() CropAndResizePlugin::initialize() CropAndResizePlugin::getSerializationSize() CropAndResizePlugin::serialize() CropAndResizePlugin::destroy() --> 0xf9e8e0 CropAndResizePlugin::~CropAndResizePlugin() CropAndResizePlugin::getSerializationSize() CropAndResizePlugin::serialize() CropAndResizePlugin::destroy() --> 0x12458e00 CropAndResizePlugin::~CropAndResizePlugin() CropAndResizePlugin::destroy() --> 0x12459300 CropAndResizePlugin::~CropAndResizePlugin() engine building done createExecutionContext and inference: CropAndResizePlugin::clone() --> 0x166b2840 CropAndResizePlugin::CropAndResizePlugin() CropAndResizePlugin::attachToContext() CropAndResizePlugin::enqueue() ropAndResizePlugin::detachFromContext CropAndResizePlugin::terminate CropAndResizePlugin::destroy() --> 0x166b2840 CropAndResizePlugin::~CropAndResizePlugin() CropAndResizePlugin::terminate() CropAndResizePlugin::destroy() --> 0x6380f750 CropAndResizePlugin::~CropAndResizePlugin() Finished
- The following shows the callback flow from engine/plan deserializing to inferencing (take reference from TensorRT 5.1.5).
Deserializing: CropAndResizePluginCreator::deserializePlugin --> 0xd10c5b0 CropAndResizePlugin::CropAndResizePlugin CropAndResizePlugin::initialize() Deserializing done createExecutionContext and infer: CropAndResizePlugin::enqueue() CropAndResizePlugin::terminate() CropAndResizePlugin::destroy() CropAndResizePlugin::~CropAndResizePlugin() Finished
NOTE: TensorRT 5.1 has a known issue that initialize() and terminate() don’t appear in pairs (terminate() will be invoked twice by different objects). If you want to allocate immutable resource for your plugin in initialize(), you better make it as std::shared_ptr so that it won’t get released twice from terminate().