Patent
Atlas literature
Patent
US 10,325,201Patent drawings and their descriptions. Click a drawing to enlarge it.
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method for generating at least one deceivable composite image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising steps of: (a) a computing device, if at least one background image to be used as a background of the rare event on the deceivable composite image and at least one rare object image of at least one rare object corresponding to the rare event are acquired, instructing at least one locating layer in the generating neural network to generate one or more location candidates of the rare object on the background image into which the rare object image is to be inserted, and, if each of candidate scores of each of the location candidates calculated by at least one first discriminator is acquired, instructing the locating layer to select a specific location candidate among the location candidates as an optimal location of the rare object by referring to the candidate scores; (b) the computing device instructing at least one compositing layer in the generating neural network to insert the rare object image into the optimal location, to thereby generate at least one initial composite image; and 30 (c) the computing device instructing at least one 5 adjusting layer in the generating neural network to adjust color values corresponding to at least part of each of pixels in the initial composite image, to thereby generate the deceivable composite image.
The method of Claim 1, wherein, at the step of (c), the computing device instructs the adjusting layer to apply at least one convolution operation and at least one deconvolution operation to at least one specific region, corresponding to at least part of the initial composite image, into which the rare object image is inserted, to thereby generate the deceivable composite image.
The method of Claim 1, wherein the method further comprises a step of: (d) the computing device, if at least part of at least one deceivability score, of the deceivable composite image, calculated by at least one second discriminator and at least one existence score, of the rare object in the deceivable composite image, calculated by an object detection network are acquired, instructing at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score and the existence score, to thereby learn at least part of one or more parameters of the generating neural network by 30 backpropagating the losses.
The method of Claim 1, wherein, at the step of (a), the locating layer (i) generates one or more background-object bounding boxes including background objects in the background image, and (ii) additionally generates each of rare-object bounding boxes corresponding to the rare object on each of the location candidates, to thereby generate each of composite layouts, on the background image, and wherein the first discriminator calculates each of the candidate scores by referring to at least part of the composite layouts.
A method for testing of generating at least one deceivable composite test image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising steps of: (a) a testing device, on condition that (1) a learning device, after acquiring at least one background training 30 image to be used as a background of the rare event on the 5 deceivable composite training image and at least one rare object training image of at least one rare object for training corresponding to the rare event, has performed processes of instructing at least one locating layer in the generating neural network to generate one or more location candidates for training of the rare object for training on the background training image into which the rare object training image is to be inserted, and after acquiring each of candidate scores for training of each of the location candidates for training calculated by at least one first discriminator, instructing the locating layer to select a specific location candidate for training among the location candidates for training as an optimal location for training of the rare object for training by referring to the candidate scores for training, (2) the learning device has instructed at least one compositing layer in the generating neural network to insert the rare object training image into the optimal location for training, to thereby generate at least one initial composite training image, (3) the learning device has instructed at least one adjusting layer in the generating neural network to adjust color values for training corresponding to at least part of each of pixels in the initial composite training image, to thereby generate the deceivable composite training image, and (4) the learning device, after acquiring at least part of at least 30 one deceivability score for training, of the deceivable 5 composite training image, calculated by at least one second discriminator and at least one existence score for training, of the rare object for training in the deceivable composite training image, calculated by an object detection network, has instructed at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score for training and the existence score for training, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses; if at least one background test image to be used as a background of the rare event on the deceivable composite test image and at least one rare object test image of at least one rare object for testing corresponding to the rare event are acquired, instructing the locating layer in the generating neural network to generate one or more location candidates for testing of the rare object for testing on the background test image into which the rare object test image is to be inserted, and if each of candidate scores for testing of each of the location candidates for testing calculated by the first discriminator is acquired, instructing the locating layer to select a specific location candidate for testing among the location candidates for testing as an optimal location for testing of the rare object for testing by referring to the candidate scores for testing; 30 (b) the testing device instructing the compositing 5 layer in the generating neural network to insert the rare object test image into the optimal location for testing, to thereby generate at least one initial composite test image; and (c) the testing device instructing the adjusting layer in the generating neural network to adjust color values for testing corresponding to at least part of each of pixels in the initial composite test image, to thereby generate the deceivable composite test image.
A computing device for generating at least one deceivable composite image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising: at least one memory that stores instructions; and at least one processor configured to execute the instructions to: perform processes of (I), if at least one background image to be used as a background of the rare event on the deceivable composite image and at least one rare object image of at least one rare object corresponding to the rare event are acquired, instructing at least one locating layer in the generating neural network to generate one or more location candidates of the rare object on the 30 background image into which the rare object image is to be inserted, and if each of candidate scores of each of the location candidates calculated by at least one first discriminator is acquired, instructing the locating layer to select a specific location candidate among the location candidates as an optimal location of the rare object by referring to the candidate scores, (II) instructing at least one compositing layer in the generating neural network to insert the rare object image into the optimal location, to thereby generate at least one initial composite image, and (III) instructing at least one adjusting layer in the generating neural network to adjust color values corresponding to at least part of each of pixels in the initial composite image, to thereby generate the deceivable composite image.
The computing device of Claim 13, wherein, at the process of (III), the processor instructs the adjusting layer to apply at least one convolution operation and at least one deconvolution operation to at least one specific region, corresponding to at least part of the initial composite image, into which the rare object image is inserted, to thereby generate the deceivable composite image.
The computing device of Claim 13, wherein the processor further performs a process of (IV), if at least part of at 30 least one deceivability score, of the deceivable composite 5 image, calculated by at least one second discriminator and at least one existence score, of the rare object in the deceivable composite image, calculated by an object detection network are acquired, instructing at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score and the existence score, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses.
The computing device of Claim 13, wherein, at the process of (I), the locating layer (i) generates one or more background-object bounding boxes including background objects in the backg roun d image, and (ii) additionally generates each of rar e-object b ounding boxes corresponding to the rare object on each of the location candidates, to thereby generate each of composite layouts, on the background image, and wherein the first discriminator calculates each of the candidate scores by referring to at least part of the composite layouts.
A testing device for testing of generating at least one deceivable composite test image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising: at least one memory that stores instructions; and at least one processor, on condition that a learning device (1), after acquiring at least one background training image to be used as a background of the rare event on the deceivable composite training image and at least one rare object training image of at least one rare object for training corresponding to the rare event, has performed processes of instructing at least one locating layer in the generating neural network to generate one or more location candidates for training of the rare object for training on the background training image into which the rare object training image is to be inserted, and after acquiring each of candidate scores for training of each of the location candidates for training calculated by at least one first discriminator, instructing the locating layer to select a specific location candidate for training among the location candidates for training as an optimal location for training of the rare object for training by referring to the 30 candidate scores for training, (2) has instructed at least 5 one compositing layer in the generating neural network to insert the rare object training image into the optimal location for training, to thereby generate at least one initial composite training image, (3) has instructed at least one adjusting layer in the generating neural network to adjust color values for training corresponding to at least part of each of pixels in the initial composite training image, to thereby generate the deceivable composite training image, and (4), after acquiring at least part of at least one deceivability score for training, of the deceivable composite training image, calculated by at least one second discriminator and at least one existence score for training, of the rare object for training in the deceivable composite training image, calculated by an object detection network, has instructed at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score for training and the existence score for training, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses; configured to execute the instructions to: perform processes of (I) if at least one background test image to be used as a background of the rare event on the deceivable composite test image and at least one rare object test image of at least one rare object for testing corresponding to the rare event are acquired, instructing the locating layer in the 5 generating neural network to generate one or more location candidates for testing of the rare object for testing on the background test image into which the rare object test image is to be inserted, and if each of candidate scores for testing of each of the location candidates for testing calculated by the first discriminator is acquired, instructing the locating layer to select a specific location candidate for testing among the location candidates for testing as an optimal location for testing of the rare object for testing by referring to the candidate scores for testing, (II) instructing the compositing layer in the generating neural network to insert the rare object test image into the optimal location for testing, to thereby generate at least one initial composite test image, and (III) instructing the adjusting layer in the generating neural network to adjust color values for testing corresponding to at least part of each of pixels in the initial composite test image, to thereby generate the deceivable composite test image.
Patent
Atlas literature
Patent
US 10,325,201Patent drawings and their descriptions. Click a drawing to enlarge it.
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method for generating at least one deceivable composite image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising steps of: (a) a computing device, if at least one background image to be used as a background of the rare event on the deceivable composite image and at least one rare object image of at least one rare object corresponding to the rare event are acquired, instructing at least one locating layer in the generating neural network to generate one or more location candidates of the rare object on the background image into which the rare object image is to be inserted, and, if each of candidate scores of each of the location candidates calculated by at least one first discriminator is acquired, instructing the locating layer to select a specific location candidate among the location candidates as an optimal location of the rare object by referring to the candidate scores; (b) the computing device instructing at least one compositing layer in the generating neural network to insert the rare object image into the optimal location, to thereby generate at least one initial composite image; and 30 (c) the computing device instructing at least one 5 adjusting layer in the generating neural network to adjust color values corresponding to at least part of each of pixels in the initial composite image, to thereby generate the deceivable composite image.
The method of Claim 1, wherein, at the step of (c), the computing device instructs the adjusting layer to apply at least one convolution operation and at least one deconvolution operation to at least one specific region, corresponding to at least part of the initial composite image, into which the rare object image is inserted, to thereby generate the deceivable composite image.
The method of Claim 1, wherein the method further comprises a step of: (d) the computing device, if at least part of at least one deceivability score, of the deceivable composite image, calculated by at least one second discriminator and at least one existence score, of the rare object in the deceivable composite image, calculated by an object detection network are acquired, instructing at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score and the existence score, to thereby learn at least part of one or more parameters of the generating neural network by 30 backpropagating the losses.
The method of Claim 1, wherein, at the step of (a), the locating layer (i) generates one or more background-object bounding boxes including background objects in the background image, and (ii) additionally generates each of rare-object bounding boxes corresponding to the rare object on each of the location candidates, to thereby generate each of composite layouts, on the background image, and wherein the first discriminator calculates each of the candidate scores by referring to at least part of the composite layouts.
A method for testing of generating at least one deceivable composite test image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising steps of: (a) a testing device, on condition that (1) a learning device, after acquiring at least one background training 30 image to be used as a background of the rare event on the 5 deceivable composite training image and at least one rare object training image of at least one rare object for training corresponding to the rare event, has performed processes of instructing at least one locating layer in the generating neural network to generate one or more location candidates for training of the rare object for training on the background training image into which the rare object training image is to be inserted, and after acquiring each of candidate scores for training of each of the location candidates for training calculated by at least one first discriminator, instructing the locating layer to select a specific location candidate for training among the location candidates for training as an optimal location for training of the rare object for training by referring to the candidate scores for training, (2) the learning device has instructed at least one compositing layer in the generating neural network to insert the rare object training image into the optimal location for training, to thereby generate at least one initial composite training image, (3) the learning device has instructed at least one adjusting layer in the generating neural network to adjust color values for training corresponding to at least part of each of pixels in the initial composite training image, to thereby generate the deceivable composite training image, and (4) the learning device, after acquiring at least part of at least 30 one deceivability score for training, of the deceivable 5 composite training image, calculated by at least one second discriminator and at least one existence score for training, of the rare object for training in the deceivable composite training image, calculated by an object detection network, has instructed at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score for training and the existence score for training, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses; if at least one background test image to be used as a background of the rare event on the deceivable composite test image and at least one rare object test image of at least one rare object for testing corresponding to the rare event are acquired, instructing the locating layer in the generating neural network to generate one or more location candidates for testing of the rare object for testing on the background test image into which the rare object test image is to be inserted, and if each of candidate scores for testing of each of the location candidates for testing calculated by the first discriminator is acquired, instructing the locating layer to select a specific location candidate for testing among the location candidates for testing as an optimal location for testing of the rare object for testing by referring to the candidate scores for testing; 30 (b) the testing device instructing the compositing 5 layer in the generating neural network to insert the rare object test image into the optimal location for testing, to thereby generate at least one initial composite test image; and (c) the testing device instructing the adjusting layer in the generating neural network to adjust color values for testing corresponding to at least part of each of pixels in the initial composite test image, to thereby generate the deceivable composite test image.
A computing device for generating at least one deceivable composite image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising: at least one memory that stores instructions; and at least one processor configured to execute the instructions to: perform processes of (I), if at least one background image to be used as a background of the rare event on the deceivable composite image and at least one rare object image of at least one rare object corresponding to the rare event are acquired, instructing at least one locating layer in the generating neural network to generate one or more location candidates of the rare object on the 30 background image into which the rare object image is to be inserted, and if each of candidate scores of each of the location candidates calculated by at least one first discriminator is acquired, instructing the locating layer to select a specific location candidate among the location candidates as an optimal location of the rare object by referring to the candidate scores, (II) instructing at least one compositing layer in the generating neural network to insert the rare object image into the optimal location, to thereby generate at least one initial composite image, and (III) instructing at least one adjusting layer in the generating neural network to adjust color values corresponding to at least part of each of pixels in the initial composite image, to thereby generate the deceivable composite image.
The computing device of Claim 13, wherein, at the process of (III), the processor instructs the adjusting layer to apply at least one convolution operation and at least one deconvolution operation to at least one specific region, corresponding to at least part of the initial composite image, into which the rare object image is inserted, to thereby generate the deceivable composite image.
The computing device of Claim 13, wherein the processor further performs a process of (IV), if at least part of at 30 least one deceivability score, of the deceivable composite 5 image, calculated by at least one second discriminator and at least one existence score, of the rare object in the deceivable composite image, calculated by an object detection network are acquired, instructing at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score and the existence score, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses.
The computing device of Claim 13, wherein, at the process of (I), the locating layer (i) generates one or more background-object bounding boxes including background objects in the backg roun d image, and (ii) additionally generates each of rar e-object b ounding boxes corresponding to the rare object on each of the location candidates, to thereby generate each of composite layouts, on the background image, and wherein the first discriminator calculates each of the candidate scores by referring to at least part of the composite layouts.
A testing device for testing of generating at least one deceivable composite test image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising: at least one memory that stores instructions; and at least one processor, on condition that a learning device (1), after acquiring at least one background training image to be used as a background of the rare event on the deceivable composite training image and at least one rare object training image of at least one rare object for training corresponding to the rare event, has performed processes of instructing at least one locating layer in the generating neural network to generate one or more location candidates for training of the rare object for training on the background training image into which the rare object training image is to be inserted, and after acquiring each of candidate scores for training of each of the location candidates for training calculated by at least one first discriminator, instructing the locating layer to select a specific location candidate for training among the location candidates for training as an optimal location for training of the rare object for training by referring to the 30 candidate scores for training, (2) has instructed at least 5 one compositing layer in the generating neural network to insert the rare object training image into the optimal location for training, to thereby generate at least one initial composite training image, (3) has instructed at least one adjusting layer in the generating neural network to adjust color values for training corresponding to at least part of each of pixels in the initial composite training image, to thereby generate the deceivable composite training image, and (4), after acquiring at least part of at least one deceivability score for training, of the deceivable composite training image, calculated by at least one second discriminator and at least one existence score for training, of the rare object for training in the deceivable composite training image, calculated by an object detection network, has instructed at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score for training and the existence score for training, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses; configured to execute the instructions to: perform processes of (I) if at least one background test image to be used as a background of the rare event on the deceivable composite test image and at least one rare object test image of at least one rare object for testing corresponding to the rare event are acquired, instructing the locating layer in the 5 generating neural network to generate one or more location candidates for testing of the rare object for testing on the background test image into which the rare object test image is to be inserted, and if each of candidate scores for testing of each of the location candidates for testing calculated by the first discriminator is acquired, instructing the locating layer to select a specific location candidate for testing among the location candidates for testing as an optimal location for testing of the rare object for testing by referring to the candidate scores for testing, (II) instructing the compositing layer in the generating neural network to insert the rare object test image into the optimal location for testing, to thereby generate at least one initial composite test image, and (III) instructing the adjusting layer in the generating neural network to adjust color values for testing corresponding to at least part of each of pixels in the initial composite test image, to thereby generate the deceivable composite test image.
Patent
Atlas literature
Patent
US 10,325,201Patent drawings and their descriptions. Click a drawing to enlarge it.
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method for generating at least one deceivable composite image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising steps of: (a) a computing device, if at least one background image to be used as a background of the rare event on the deceivable composite image and at least one rare object image of at least one rare object corresponding to the rare event are acquired, instructing at least one locating layer in the generating neural network to generate one or more location candidates of the rare object on the background image into which the rare object image is to be inserted, and, if each of candidate scores of each of the location candidates calculated by at least one first discriminator is acquired, instructing the locating layer to select a specific location candidate among the location candidates as an optimal location of the rare object by referring to the candidate scores; (b) the computing device instructing at least one compositing layer in the generating neural network to insert the rare object image into the optimal location, to thereby generate at least one initial composite image; and 30 (c) the computing device instructing at least one 5 adjusting layer in the generating neural network to adjust color values corresponding to at least part of each of pixels in the initial composite image, to thereby generate the deceivable composite image.
The method of Claim 1, wherein, at the step of (c), the computing device instructs the adjusting layer to apply at least one convolution operation and at least one deconvolution operation to at least one specific region, corresponding to at least part of the initial composite image, into which the rare object image is inserted, to thereby generate the deceivable composite image.
The method of Claim 1, wherein the method further comprises a step of: (d) the computing device, if at least part of at least one deceivability score, of the deceivable composite image, calculated by at least one second discriminator and at least one existence score, of the rare object in the deceivable composite image, calculated by an object detection network are acquired, instructing at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score and the existence score, to thereby learn at least part of one or more parameters of the generating neural network by 30 backpropagating the losses.
The method of Claim 1, wherein, at the step of (a), the locating layer (i) generates one or more background-object bounding boxes including background objects in the background image, and (ii) additionally generates each of rare-object bounding boxes corresponding to the rare object on each of the location candidates, to thereby generate each of composite layouts, on the background image, and wherein the first discriminator calculates each of the candidate scores by referring to at least part of the composite layouts.
A method for testing of generating at least one deceivable composite test image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising steps of: (a) a testing device, on condition that (1) a learning device, after acquiring at least one background training 30 image to be used as a background of the rare event on the 5 deceivable composite training image and at least one rare object training image of at least one rare object for training corresponding to the rare event, has performed processes of instructing at least one locating layer in the generating neural network to generate one or more location candidates for training of the rare object for training on the background training image into which the rare object training image is to be inserted, and after acquiring each of candidate scores for training of each of the location candidates for training calculated by at least one first discriminator, instructing the locating layer to select a specific location candidate for training among the location candidates for training as an optimal location for training of the rare object for training by referring to the candidate scores for training, (2) the learning device has instructed at least one compositing layer in the generating neural network to insert the rare object training image into the optimal location for training, to thereby generate at least one initial composite training image, (3) the learning device has instructed at least one adjusting layer in the generating neural network to adjust color values for training corresponding to at least part of each of pixels in the initial composite training image, to thereby generate the deceivable composite training image, and (4) the learning device, after acquiring at least part of at least 30 one deceivability score for training, of the deceivable 5 composite training image, calculated by at least one second discriminator and at least one existence score for training, of the rare object for training in the deceivable composite training image, calculated by an object detection network, has instructed at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score for training and the existence score for training, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses; if at least one background test image to be used as a background of the rare event on the deceivable composite test image and at least one rare object test image of at least one rare object for testing corresponding to the rare event are acquired, instructing the locating layer in the generating neural network to generate one or more location candidates for testing of the rare object for testing on the background test image into which the rare object test image is to be inserted, and if each of candidate scores for testing of each of the location candidates for testing calculated by the first discriminator is acquired, instructing the locating layer to select a specific location candidate for testing among the location candidates for testing as an optimal location for testing of the rare object for testing by referring to the candidate scores for testing; 30 (b) the testing device instructing the compositing 5 layer in the generating neural network to insert the rare object test image into the optimal location for testing, to thereby generate at least one initial composite test image; and (c) the testing device instructing the adjusting layer in the generating neural network to adjust color values for testing corresponding to at least part of each of pixels in the initial composite test image, to thereby generate the deceivable composite test image.
A computing device for generating at least one deceivable composite image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising: at least one memory that stores instructions; and at least one processor configured to execute the instructions to: perform processes of (I), if at least one background image to be used as a background of the rare event on the deceivable composite image and at least one rare object image of at least one rare object corresponding to the rare event are acquired, instructing at least one locating layer in the generating neural network to generate one or more location candidates of the rare object on the 30 background image into which the rare object image is to be inserted, and if each of candidate scores of each of the location candidates calculated by at least one first discriminator is acquired, instructing the locating layer to select a specific location candidate among the location candidates as an optimal location of the rare object by referring to the candidate scores, (II) instructing at least one compositing layer in the generating neural network to insert the rare object image into the optimal location, to thereby generate at least one initial composite image, and (III) instructing at least one adjusting layer in the generating neural network to adjust color values corresponding to at least part of each of pixels in the initial composite image, to thereby generate the deceivable composite image.
The computing device of Claim 13, wherein, at the process of (III), the processor instructs the adjusting layer to apply at least one convolution operation and at least one deconvolution operation to at least one specific region, corresponding to at least part of the initial composite image, into which the rare object image is inserted, to thereby generate the deceivable composite image.
The computing device of Claim 13, wherein the processor further performs a process of (IV), if at least part of at 30 least one deceivability score, of the deceivable composite 5 image, calculated by at least one second discriminator and at least one existence score, of the rare object in the deceivable composite image, calculated by an object detection network are acquired, instructing at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score and the existence score, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses.
The computing device of Claim 13, wherein, at the process of (I), the locating layer (i) generates one or more background-object bounding boxes including background objects in the backg roun d image, and (ii) additionally generates each of rar e-object b ounding boxes corresponding to the rare object on each of the location candidates, to thereby generate each of composite layouts, on the background image, and wherein the first discriminator calculates each of the candidate scores by referring to at least part of the composite layouts.
A testing device for testing of generating at least one deceivable composite test image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising: at least one memory that stores instructions; and at least one processor, on condition that a learning device (1), after acquiring at least one background training image to be used as a background of the rare event on the deceivable composite training image and at least one rare object training image of at least one rare object for training corresponding to the rare event, has performed processes of instructing at least one locating layer in the generating neural network to generate one or more location candidates for training of the rare object for training on the background training image into which the rare object training image is to be inserted, and after acquiring each of candidate scores for training of each of the location candidates for training calculated by at least one first discriminator, instructing the locating layer to select a specific location candidate for training among the location candidates for training as an optimal location for training of the rare object for training by referring to the 30 candidate scores for training, (2) has instructed at least 5 one compositing layer in the generating neural network to insert the rare object training image into the optimal location for training, to thereby generate at least one initial composite training image, (3) has instructed at least one adjusting layer in the generating neural network to adjust color values for training corresponding to at least part of each of pixels in the initial composite training image, to thereby generate the deceivable composite training image, and (4), after acquiring at least part of at least one deceivability score for training, of the deceivable composite training image, calculated by at least one second discriminator and at least one existence score for training, of the rare object for training in the deceivable composite training image, calculated by an object detection network, has instructed at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score for training and the existence score for training, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses; configured to execute the instructions to: perform processes of (I) if at least one background test image to be used as a background of the rare event on the deceivable composite test image and at least one rare object test image of at least one rare object for testing corresponding to the rare event are acquired, instructing the locating layer in the 5 generating neural network to generate one or more location candidates for testing of the rare object for testing on the background test image into which the rare object test image is to be inserted, and if each of candidate scores for testing of each of the location candidates for testing calculated by the first discriminator is acquired, instructing the locating layer to select a specific location candidate for testing among the location candidates for testing as an optimal location for testing of the rare object for testing by referring to the candidate scores for testing, (II) instructing the compositing layer in the generating neural network to insert the rare object test image into the optimal location for testing, to thereby generate at least one initial composite test image, and (III) instructing the adjusting layer in the generating neural network to adjust color values for testing corresponding to at least part of each of pixels in the initial composite test image, to thereby generate the deceivable composite test image.
Patent
Atlas literature
Patent
US 10,325,201Patent drawings and their descriptions. Click a drawing to enlarge it.
Claims define the patent's legal scope. Independent claims stand alone; dependent claims (nested) narrow them. Click a claim to expand its dependents.
A method for generating at least one deceivable composite image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising steps of: (a) a computing device, if at least one background image to be used as a background of the rare event on the deceivable composite image and at least one rare object image of at least one rare object corresponding to the rare event are acquired, instructing at least one locating layer in the generating neural network to generate one or more location candidates of the rare object on the background image into which the rare object image is to be inserted, and, if each of candidate scores of each of the location candidates calculated by at least one first discriminator is acquired, instructing the locating layer to select a specific location candidate among the location candidates as an optimal location of the rare object by referring to the candidate scores; (b) the computing device instructing at least one compositing layer in the generating neural network to insert the rare object image into the optimal location, to thereby generate at least one initial composite image; and 30 (c) the computing device instructing at least one 5 adjusting layer in the generating neural network to adjust color values corresponding to at least part of each of pixels in the initial composite image, to thereby generate the deceivable composite image.
The method of Claim 1, wherein, at the step of (c), the computing device instructs the adjusting layer to apply at least one convolution operation and at least one deconvolution operation to at least one specific region, corresponding to at least part of the initial composite image, into which the rare object image is inserted, to thereby generate the deceivable composite image.
The method of Claim 1, wherein the method further comprises a step of: (d) the computing device, if at least part of at least one deceivability score, of the deceivable composite image, calculated by at least one second discriminator and at least one existence score, of the rare object in the deceivable composite image, calculated by an object detection network are acquired, instructing at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score and the existence score, to thereby learn at least part of one or more parameters of the generating neural network by 30 backpropagating the losses.
The method of Claim 1, wherein, at the step of (a), the locating layer (i) generates one or more background-object bounding boxes including background objects in the background image, and (ii) additionally generates each of rare-object bounding boxes corresponding to the rare object on each of the location candidates, to thereby generate each of composite layouts, on the background image, and wherein the first discriminator calculates each of the candidate scores by referring to at least part of the composite layouts.
A method for testing of generating at least one deceivable composite test image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising steps of: (a) a testing device, on condition that (1) a learning device, after acquiring at least one background training 30 image to be used as a background of the rare event on the 5 deceivable composite training image and at least one rare object training image of at least one rare object for training corresponding to the rare event, has performed processes of instructing at least one locating layer in the generating neural network to generate one or more location candidates for training of the rare object for training on the background training image into which the rare object training image is to be inserted, and after acquiring each of candidate scores for training of each of the location candidates for training calculated by at least one first discriminator, instructing the locating layer to select a specific location candidate for training among the location candidates for training as an optimal location for training of the rare object for training by referring to the candidate scores for training, (2) the learning device has instructed at least one compositing layer in the generating neural network to insert the rare object training image into the optimal location for training, to thereby generate at least one initial composite training image, (3) the learning device has instructed at least one adjusting layer in the generating neural network to adjust color values for training corresponding to at least part of each of pixels in the initial composite training image, to thereby generate the deceivable composite training image, and (4) the learning device, after acquiring at least part of at least 30 one deceivability score for training, of the deceivable 5 composite training image, calculated by at least one second discriminator and at least one existence score for training, of the rare object for training in the deceivable composite training image, calculated by an object detection network, has instructed at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score for training and the existence score for training, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses; if at least one background test image to be used as a background of the rare event on the deceivable composite test image and at least one rare object test image of at least one rare object for testing corresponding to the rare event are acquired, instructing the locating layer in the generating neural network to generate one or more location candidates for testing of the rare object for testing on the background test image into which the rare object test image is to be inserted, and if each of candidate scores for testing of each of the location candidates for testing calculated by the first discriminator is acquired, instructing the locating layer to select a specific location candidate for testing among the location candidates for testing as an optimal location for testing of the rare object for testing by referring to the candidate scores for testing; 30 (b) the testing device instructing the compositing 5 layer in the generating neural network to insert the rare object test image into the optimal location for testing, to thereby generate at least one initial composite test image; and (c) the testing device instructing the adjusting layer in the generating neural network to adjust color values for testing corresponding to at least part of each of pixels in the initial composite test image, to thereby generate the deceivable composite test image.
A computing device for generating at least one deceivable composite image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising: at least one memory that stores instructions; and at least one processor configured to execute the instructions to: perform processes of (I), if at least one background image to be used as a background of the rare event on the deceivable composite image and at least one rare object image of at least one rare object corresponding to the rare event are acquired, instructing at least one locating layer in the generating neural network to generate one or more location candidates of the rare object on the 30 background image into which the rare object image is to be inserted, and if each of candidate scores of each of the location candidates calculated by at least one first discriminator is acquired, instructing the locating layer to select a specific location candidate among the location candidates as an optimal location of the rare object by referring to the candidate scores, (II) instructing at least one compositing layer in the generating neural network to insert the rare object image into the optimal location, to thereby generate at least one initial composite image, and (III) instructing at least one adjusting layer in the generating neural network to adjust color values corresponding to at least part of each of pixels in the initial composite image, to thereby generate the deceivable composite image.
The computing device of Claim 13, wherein, at the process of (III), the processor instructs the adjusting layer to apply at least one convolution operation and at least one deconvolution operation to at least one specific region, corresponding to at least part of the initial composite image, into which the rare object image is inserted, to thereby generate the deceivable composite image.
The computing device of Claim 13, wherein the processor further performs a process of (IV), if at least part of at 30 least one deceivability score, of the deceivable composite 5 image, calculated by at least one second discriminator and at least one existence score, of the rare object in the deceivable composite image, calculated by an object detection network are acquired, instructing at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score and the existence score, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses.
The computing device of Claim 13, wherein, at the process of (I), the locating layer (i) generates one or more background-object bounding boxes including background objects in the backg roun d image, and (ii) additionally generates each of rar e-object b ounding boxes corresponding to the rare object on each of the location candidates, to thereby generate each of composite layouts, on the background image, and wherein the first discriminator calculates each of the candidate scores by referring to at least part of the composite layouts.
A testing device for testing of generating at least one deceivable composite test image by using a GAN (Generative Adversarial Network) including a generating neural network and a discriminating neural network to allow a surveillance system to detect at least one rare event more accurately, comprising: at least one memory that stores instructions; and at least one processor, on condition that a learning device (1), after acquiring at least one background training image to be used as a background of the rare event on the deceivable composite training image and at least one rare object training image of at least one rare object for training corresponding to the rare event, has performed processes of instructing at least one locating layer in the generating neural network to generate one or more location candidates for training of the rare object for training on the background training image into which the rare object training image is to be inserted, and after acquiring each of candidate scores for training of each of the location candidates for training calculated by at least one first discriminator, instructing the locating layer to select a specific location candidate for training among the location candidates for training as an optimal location for training of the rare object for training by referring to the 30 candidate scores for training, (2) has instructed at least 5 one compositing layer in the generating neural network to insert the rare object training image into the optimal location for training, to thereby generate at least one initial composite training image, (3) has instructed at least one adjusting layer in the generating neural network to adjust color values for training corresponding to at least part of each of pixels in the initial composite training image, to thereby generate the deceivable composite training image, and (4), after acquiring at least part of at least one deceivability score for training, of the deceivable composite training image, calculated by at least one second discriminator and at least one existence score for training, of the rare object for training in the deceivable composite training image, calculated by an object detection network, has instructed at least one loss layer in the generating neural network to calculate one or more losses by referring to at least part of the deceivability score for training and the existence score for training, to thereby learn at least part of one or more parameters of the generating neural network by backpropagating the losses; configured to execute the instructions to: perform processes of (I) if at least one background test image to be used as a background of the rare event on the deceivable composite test image and at least one rare object test image of at least one rare object for testing corresponding to the rare event are acquired, instructing the locating layer in the 5 generating neural network to generate one or more location candidates for testing of the rare object for testing on the background test image into which the rare object test image is to be inserted, and if each of candidate scores for testing of each of the location candidates for testing calculated by the first discriminator is acquired, instructing the locating layer to select a specific location candidate for testing among the location candidates for testing as an optimal location for testing of the rare object for testing by referring to the candidate scores for testing, (II) instructing the compositing layer in the generating neural network to insert the rare object test image into the optimal location for testing, to thereby generate at least one initial composite test image, and (III) instructing the adjusting layer in the generating neural network to adjust color values for testing corresponding to at least part of each of pixels in the initial composite test image, to thereby generate the deceivable composite test image.
