a +caP@sRdZddlZddlZddlZddlZddlZddlZ ddl Z ddl Z ddl Z ddlZddlZddlmZddZddZdd Zd d Zd d ZddZddZddZGdddeZd?ddZd@ddZddZddZdd Z d!d"Z!d#d$Z"d%d&Z#d'd(Z$dAd+d,Z%dBd-d.Z&dCd0d1Z'd2d3Z(dDd4d5Z)d6d7Z*d8d9Z+dEd=d>Z,dS)Fa Mask R-CNN Common utility functions and classes. Copyright (c) 2017 Matterport, Inc. Licensed under the MIT License (see LICENSE for details) Written by Waleed Abdulla Modified by Ondrej Pesek Written import_contents and load_mask methods Different way of removing the alpha channel (contained in load_mask) Deleted download_train_weights A little refactoring made # last cross-check with Matterport's implementation: 28/08/2018 N) LooseVersionc Cstj|jddgtjd}t|jdD]}|dddd|f}ttj|ddd}ttj|ddd}|jdr|ddg\}}|ddg\}} |d7}| d7} n d\}}}} t||| |g||<q(|tjS) zCompute bounding boxes from masks. mask: [height, width, num_instances]. Mask pixels are either 1 or 0. Returns: bbox array [num_instances, (y1, x1, y2, x2)]. dtypeNraxis)rrrr) npzerosshapeint32rangewhereanyarrayastype) maskboxesimZhorizontal_indiciesZvertical_indiciesx1x2y1y2rUC:/Users/landamar/grass_packager/grass786/addons/i.ann.maskrcnn/etc/maskrcnn/utils.pyextract_bboxes&s   rc Cst|d|dddf}t|d|dddf}t|d|dddf}t|d|dddf}t||dt||d}||dd|dd} || } | S)asCalculates IoU of the given box with the array of the given boxes. box: 1D vector [y1, x1, y2, x2] boxes: [boxes_count, (y1, x1, y2, x2)] box_area: float. the area of 'box' boxes_area: array of length boxes_count. Note: the areas are passed in rather than calculated here for efficiency. Calculate once in the caller to avoid duplicate work. rNr )r maximumZminimum) boxrZbox_areaZ boxes_arearrrr intersectionunioniourrr compute_iou@s  r%cCs|dddf|dddf|dddf|dddf}|dddf|dddf|dddf|dddf}t|jd|jdf}t|jdD]*}||}t||||||dd|f<q|S)zComputes IoU overlaps between two sets of boxes. boxes1, boxes2: [N, (y1, x1, y2, x2)]. For better performance, pass the largest set first and the smaller second. Nrrrr )r r r rr%)Zboxes1Zboxes2area1area2overlapsrZbox2rrrcompute_overlapsUs@@ r)cCs|jddks|jddkr6t|jd|jdfSt|dkd|jdftj}t|dkd|jdftj}tj|dd}tj|dd}t|j|}|dddf|dddf|}||}|S)zdComputes IoU overlaps between two sets of masks. masks1, masks2: [Height, Width, instances] rr?rN) r r r reshaperfloat32sumdotT)Zmasks1Zmasks2r&r'Z intersectionsr#r(rrrcompute_overlaps_maskshs""$r0c Cs|jddksJ|jjdkr*|tj}|dddf}|dddf}|dddf}|dddf}||||}|ddd}g} t|dkr|d} | | t || ||dd|| ||dd} t | |kdd} t || }t |d}qtj | tj dS) zPerforms non-maximum suppression and returns indices of kept boxes. boxes: [N, (y1, x1, y2, x2)]. Notice that (y2, x2) lays outside the box. scores: 1-D array of box scores. threshold: Float. IoU threshold to use for filtering. rfNr rrrr)r rkindrr r,argsortlenappendr%rdeleterr ) rZscores thresholdrrrrZareaZixsZpickrr$Z remove_ixsrrrnon_max_suppression~s$   . r8c Cs|tj}|dddf|dddf}|dddf|dddf}|dddfd|}|dddfd|}||dddf|7}||dddf|7}|t|dddf9}|t|dddf9}|d|}|d|}||}||} tj|||| gddS)zApplies the given deltas to the given boxes. boxes: [N, (y1, x1, y2, x2)]. Note that (y2, x2) is outside the box. deltas: [N, (dy, dx, log(dh), log(dw))] Nrrrr r*r)rr r,Zexpstack) rZdeltasheightwidthcenter_ycenter_xrrrrrrrapply_box_deltass     r>cCsJt|tj}t|tj}|dddf|dddf}|dddf|dddf}|dddfd|}|dddfd|}|dddf|dddf}|dddf|dddf}|dddfd|}|dddfd|} |||} | ||} t||} t||} tj| | | | gdd}|S)zgCompute refinement needed to transform box to gt_box. box and gt_box are [N, (y1, x1, y2, x2)] Nrrrr r*r)tfcastr,logr9)r!gt_boxr:r;r<r= gt_heightgt_width gt_center_y gt_center_xdydxdhdwresultrrrbox_refinement_graphs       rLcCsB|tj}|tj}|dddf|dddf}|dddf|dddf}|dddfd|}|dddfd|}|dddf|dddf}|dddf|dddf}|dddfd|}|dddfd|} |||} | ||} t||} t||} tj| | | | gddS)zCompute refinement needed to transform box to gt_box. box and gt_box are [N, (y1, x1, y2, x2)]. (y2, x2) is assumed to be outside the box. Nrrrr r*r)rr r,rAr9)r!rBr:r;r<r=rCrDrErFrGrHrIrJrrrbox_refinements        rMc@sxeZdZdZdddZddZddZd d Zdd d Zd dZ ddZ e ddZ ddZ ddZddZddZdS)Datasetz- The base class for dataset classes. NcCs&g|_g|_ddddg|_i|_dS)NrBGsourceidname) _image_ids image_info class_infosource_class_ids)self class_maprrr__init__szDataset.__init__cCsRd|vsJd|jD]"}|d|kr|d|krdSq|j|||ddS)N.z Source name cannot contain a dotrRrSrQ)rWr5)rYrRclass_id class_nameinforrr add_classs zDataset.add_classcKs&|||d}|||j|dS)N)rSrRpath)updaterVr5)rYrRimage_idrakwargsrVrrr add_images  zDataset.add_imagecCsdS)zReturn a link to the image in its source Website or details about the image that help looking it up or debugging it. Override for your dataset, but pass to this function if you encounter images not in your dataset. rOrrYrcrrrimage_referenceszDataset.image_referencecsddt|j|_t|j|_fdd|jD|_t|j|_t|j|_ ddt |j|jD|_ ddt |j|j D|_ ttdd|jD|_i|_|jD]F}g|j|<t|jD],\}}|d ks||d kr|j||qqd S) zPrepares the Dataset class for use. TODO: class map is not supported yet. When done, it should handle mapping classes from different datasets to the same class ID. cSsd|dddS)z>Returns a shorter version of object names for cleaner display.,Nr )joinsplitrTrrr clean_name)sz#Dataset.prepare..clean_namecsg|]}|dqSrkr).0crlrr 0z#Dataset.prepare..cSs&i|]\}}d|d|d|qSz{}.{}rRrSformatrmr_rSrrr 5sz#Dataset.prepare..cSs&i|]\}}d|d|d|qSrrrsrurrrrv9scSsg|] }|dqS)rRr)rmrrrrrp?rqrrRN)r4rWZ num_classesr arange class_idsZ class_namesrVZ num_imagesrUzipclass_from_source_map image_idsZimage_from_source_maplistsetZsourcesrX enumerater5)rYrZrRrr_rrorprepare"s&      zDataset.preparecCs |j|S)zTakes a source class ID and returns the int class ID assigned to it. For example: dataset.map_source_class_id("coco.12") -> 23 )rz)rYZsource_class_idrrrmap_source_class_idJszDataset.map_source_class_idcCs"|j|}|d|ksJ|dS)zMMap an internal class ID to the corresponding class ID in the source dataset.rRrS)rW)rYr]rRr_rrrget_source_class_idRs zDataset.get_source_class_idcCs|jSN)rU)rYrrrr{XszDataset.image_idscCs|j|dS)zReturns the path or URL to the image. Override this to return a URL to the image if it's available online for easy debugging. ra)rVrfrrrsource_image_link\szDataset.source_image_linkcCs0tj|j|d}|jdkr,tj|}|S)zLoad the specified image and return a [H,W,3] Numpy array. Modified by Ondrej Pesek to not care about alpha channel rar)skimageioimreadrVndimZcolorZgray2rgb)rYrcimagerrr load_imagecs  zDataset.load_imagecCsddi|_tdt|dD]2}|||||d|j||d|iq|D]:}ttj |dD] }|j |tj |d|dqlqTdS)a  Initialize an empty Dataset class with classes and images :param classes: List of classes names :param directories: List of directories containing training images :param modelName: Name of model Written by Ondrej Pesek rPrr z*.jpg)rcraN) classesrr4r`rbglobZiglobosrarirerj)rYrZ directoriesZ modelNamerZ directoryrrrrimport_contentsos zDataset.import_contentsc Cs|j|}ttjtj|ddd}zBtj|d}t t |dkrX|}n|dddddf}WnYdS0t |j d|j ddg}t |j d|j ddg}t |j|ddd g}|t j|dddddf<tdt |D]} t ||j|| dd zxtj|d}t t |dkrn|t j|dddddf<n0|dddddft j|dddddf<WnYdS0t ||fdq||dfS) aLoad instance masks for the given image. This function converts the different mask format to one format in the form of an array of binary masks of shape [height, width, instances]. Returns: masks: A bool array of shape [height, width, instance count] with a binary mask per instance. class_ids: a 1D array of class IDs of the instance masks. Written by Ondrej Pesek rarz*.pngrN)NNr r -)rVrrrarirjrrrr4r r r rrrboolrr5 concatenate) rYrcr_aZ loadedImageZ maskImagerZ maskAppendrxrrrrget_masks0 $   4 zDataset.get_mask)N)N)__name__ __module__ __qualname____doc__r[r`rergrrrpropertyr{rrrrrrrrrNs   (  rNsquarecCs|j}|jdd\}}dd||f}d} gd} d} |dkrJ||| | | fS|rbtd|t||} |rr| |krr|} |r|dkrt||} t| | |kr|| } | dkrt|t|| t|| fdd }|dkrR|jdd\}}||d} ||| }||d}|||}| |f||fd g} tj|| d dd }| ||| ||f}nl|d kr>|jdd\}}|ddksJd|ddkr||dd}||d} ||| }nd} }|ddkr||dd}||d}|||}nd}}| |f||fd g} tj|| d dd }| ||| ||f}n|dkr|jdd\}}t d||}t d||}||||f} |||||||f}dd||f}nt d || ||| | | fS)ahResizes an image keeping the aspect ratio unchanged. min_dim: if provided, resizes the image such that it's smaller dimension == min_dim max_dim: if provided, ensures that the image longest side doesn't exceed this value. min_scale: if provided, ensure that the image is scaled up by at least this percent even if min_dim doesn't require it. mode: Resizing mode. none: No resizing. Return the image unchanged. square: Resize and pad with zeros to get a square image of size [max_dim, max_dim]. pad64: Pads width and height with zeros to make them multiples of 64. If min_dim or min_scale are provided, it scales the image up before padding. max_dim is ignored in this mode. The multiple of 64 is needed to ensure smooth scaling of feature maps up and down the 6 levels of the FPN pyramid (2**6=64). crop: Picks random crops from the image. First, scales the image based on min_dim and min_scale, then picks a random crop of size min_dim x min_dim. Can be used in training only. max_dim is not used in this mode. Returns: image: the resized image window: (y1, x1, y2, x2). If max_dim is provided, padding might be inserted in the returned image. If so, this window is the coordinates of the image part of the full image (excluding the padding). The x2, y2 pixels are not included. scale: The scale factor used to resize the image padding: Padding added to the image [(top, bottom), (left, right), (0, 0)] Nrrr )rrrrZnonerT)preserve_rangerconstantmodeZconstant_valuesZpad64@z*Minimum dimension must be a multiple of 64cropzMode {} not supported) rr maxminroundresizer padrandomZrandint Exceptionrtr)rZmin_dimZmax_dimZ min_scalerZ image_dtypehwZwindowscalepaddingrZ image_maxZtop_padZ bottom_padZleft_padZ right_padZmax_hZmax_wyxrrr resize_imagesh!    "          rcCst2tdtjj|||dgdd}Wdn1s@0Y|dur||\}}}}|||||||f}ntj||ddd}|S)aDResizes a mask using the given scale and padding. Typically, you get the scale and padding from resize_image() to ensure both, the image and the mask, are resized consistently. scale: mask scaling factor padding: Padding to add to the mask in the form [(top, bottom), (left, right), (0, 0)] ignorer r)zoomorderNrr)warningscatch_warnings simplefilterscipyZndimagerr r)rrrrrrrrrrr resize_masks  6 rc Cstj||jdftd}t|jdD]}|dddd|ft}||dd\}}}} ||||| f}|jdkrtdt||}t |tj|dddd|f<q(|S)zResize masks to a smaller version to reduce memory load. Mini-masks can be resized back to image scale using expand_masks() See inspect_data.ipynb notebook for more details. rrNrrz&Invalid bounding box with area of zero) r r r rrrsizerraround) bboxrZ mini_shape mini_maskrrrrrrrrr minimize_mask5s  &rc Cstj|dd|jdftd}t|jdD]t}|dddd|f}||dd\}}}} ||} | |} t|| | f}t|tj||||| |f<q0|S)zResizes mini masks back to image size. Reverses the change of minimize_mask(). See inspect_data.ipynb notebook for more details. Nrrrr)r r r rrrrr) rr image_shaperrrrrrrrrrrr expand_maskIs"&rcCsdSrr)rZconfigrrr mold_mask\src Cspd}|\}}}}t|||||f}t||kddtj}tj|ddtjd}||||||f<|S)a(Converts a mask generated by the neural network to a format similar to its original shape. mask: [height, width] of type float. A small, typically 28x28 mask. bbox: [y1, x1, y2, x2]. The box to fit the mask in. Returns a binary mask with the same size as the original image. r*r rNrr)rr rrrr ) rrrr7rrrrZ full_maskrrr unmold_mask`s rcCstt|t|\}}|}|}|t|}|t|}td|d||}td|d||}t||\}}t||\} } t||\} } tj| | gddddg} tj| | gddddg}tj| d|| d|gdd}|S)a scales: 1D array of anchor sizes in pixels. Example: [32, 64, 128] ratios: 1D array of anchor ratios of width/height. Example: [0.5, 1, 2] shape: [height, width] spatial shape of the feature map over which to generate anchors. feature_stride: Stride of the feature map relative to the image in pixels. anchor_stride: Stride of anchors on the feature map. For example, if the value is 2 then generate anchors for every other feature map pixel. rr rrrr*) r ZmeshgridrZflattenZsqrtrwr9r+r)scalesratiosr Zfeature_stride anchor_strideZheightsZwidthsZshifts_yZshifts_xZ box_widthsZ box_centers_xZ box_heightsZ box_centers_yZ box_centersZ box_sizesrrrrgenerate_anchorsxs rc CsFg}tt|D]&}|t||||||||qtj|ddS)aGenerate anchors at different levels of a feature pyramid. Each scale is associated with a level of the pyramid, but each ratio is used in all levels of the pyramid. Returns: anchors: [N, (y1, x1, y2, x2)]. All generated anchors in one array. Sorted with the same order of the given scales. So, anchors of scale[0] come first, then anchors of scale[1], and so on. rr)rr4r5rr r)rrZfeature_shapesZfeature_stridesrZanchorsrrrrgenerate_pyramid_anchorssrcCs*t|jdksJ|tj|dkddS)zIt's common to have tensors larger than the available data and pad with zeros. This function removes rows that are all zeros. x: [rows, columns]. rrr r)r4r r all)rrrr trim_zerossrr*c Cszt|}|dd|jdf}t|}|d|jd}t|ddd} || }|| }|| }|d| f}t||} d} dt|jdg} dt|jdg} tt|D]}t| |ddd}t| ||f|kd}|j dkr|d|d}|D]`}| |dkr"q | ||f}||kr<q||||kr | d7} || |<|| |<qq q| | | fS)aqFinds matches between prediction and ground truth instances. Returns: gt_match: 1-D array. For each GT box it has the index of the matched predicted box. pred_match: 1-D array. For each predicted box, it has the index of the matched ground truth box. overlaps: [pred_boxes, gt_boxes] IoU overlaps. .Nrrr ) rr r r3r0Zonesrr4rr)gt_boxes gt_class_idsgt_masks pred_boxespred_class_ids pred_scores pred_masks iou_thresholdZscore_thresholdindicesr(Z match_count pred_matchgt_matchrZ sorted_ixsZ low_score_idxjr$rrrcompute_matchess<      rc Cs t||||||||\}} } t| dktt| d} t| dktjt|} tdg| dgg} tdg| dgg} tt| dddD] } t | | | | d| | <qt | dd| ddkdd}t | || |d| |}|| | | fS)a5Compute Average Precision at a set IoU threshold (default 0.5). Returns: mAP: Mean Average Precision precisions: List of precisions at different class score thresholds. recalls: List of recall values at different class score thresholds. overlaps: [pred_boxes, gt_boxes] IoU overlaps. rr rrN) rr Zcumsumrwr4rr,rrr rr-)rrrrrrrrrrr( precisionsrecallsrrZmAPrrr compute_ap s& &"rr c  Cs|ptddd}g} |D]B} t|||||||| d\} } } }|rRtd| | | | qt| } |rtd|d|d| | S) zECompute AP over a range or IoU thresholds. Default range is 0.5-0.95.r*g?g?)rzAP @{:.2f}: {:.3f}zAP @{:.2f}-{:.2f}: {:.3f}rr)r rwrprintrtr5rZmean)rBZ gt_class_idZgt_maskZpred_boxZ pred_class_idZ pred_scoreZ pred_maskZiou_thresholdsverboseZAPrZaprrr(rrrcompute_ap_range<s0   rc Cs^t||}tj|dd}tj|dd}t||kd}||}tt||jd}||fS)zCompute the recall at the given IoU threshold. It's an indication of how many GT boxes were found by the given prediction boxes. pred_boxes: [N, (y1, x1, y2, x2)] in image coordinates gt_boxes: [N, (y1, x1, y2, x2)] in image coordinates r rr)r)r rZargmaxrr4r}r ) rrr$r(Ziou_maxZ iou_argmaxZ positive_idsZmatched_gt_boxesZrecallrrrcompute_recalles rcst|ts|g}g}t|D]<fdd|D}||}t|ttfsN|g}||qtt|}|dur|dgt|}ddt||D}t|dkr|d}|S)aSplits inputs into slices and feeds each slice to a copy of the given computation graph and then combines the results. It allows you to run a graph on a batch of inputs even if the graph is written to support one instance only. inputs: list of tensors. All must have the same first dimension length graph_fn: A function that returns a TF tensor that's part of a graph. batch_size: number of slices to divide the data into. names: If provided, assigns names to the resulting tensors. csg|] }|qSrr)rmrrrrrprqzbatch_slice..NcSs g|]\}}tj|d|dqS)r)rrT)r?r9)rmonrrrrprqr r) isinstancer|rtupler5ryr4)ZinputsZgraph_fnZ batch_sizenamesZoutputsZ inputs_sliceZ output_slicerKrrr batch_slice~s     rcCsP|\}}t|d|d|d|dg}tgd}t|||tjS)aaConverts boxes from pixel coordinates to normalized coordinates. boxes: [N, (y1, x1, y2, x2)] in pixel coordinates shape: [..., (height, width)] in pixels Note: In pixel coordinates (y2, x2) is outside the box. But in normalized coordinates it's inside the box. Returns: [N, (y1, x1, y2, x2)] in normalized coordinates r rrr r )r rZdividerr,rr rrrshiftrrr norm_boxess "rcCsV|\}}t|d|d|d|dg}tgd}tt|||tjS)aaConverts boxes from normalized coordinates to pixel coordinates. boxes: [N, (y1, x1, y2, x2)] in normalized coordinates shape: [..., (height, width)] in pixels Note: In pixel coordinates (y2, x2) is outside the box. But in normalized coordinates it's inside the box. Returns: [N, (y1, x1, y2, x2)] in pixel coordinates r r)r rrZmultiplyrr rrrr denorm_boxess "rrTFc CsNttjtdkr0tjj|||||||||d Stjj|||||||dSdS)aXA wrapper for Scikit-Image resize(). Scikit-Image generates warnings on every call to resize() if it doesn't receive the right parameters. The right parameters depend on the version of skimage. This solves the problem by using different parameters per version. And it provides a central place to control resizing defaults. z0.14)rrcvalclipr anti_aliasinganti_aliasing_sigma)rrrrrN)rr __version__Z transformr) rZ output_shaperrrrrrrrrrrs* r)NNNr)N)r*r)r*)Nr )N)r rrTFFN)-rrrZnumpyr Z tensorflowr?rZ skimage.colorrZ skimage.ioZskimage.transformZurllib.requestZurllibrrZdistutils.versionrrr%r)r0r8r>rLrMobjectrNrrrrrrrrrrrrrrrrrrrrrsf %B j ' K ; ) $