JustPaste.it - Share Text & Images the Easy Way

alban@alban-GL502VSK:~$ cle-nginx && cle-start
* Restarting nginx nginx [ OK ]
[1] 14334
... logging to /home/alban/.ros/log/f9874628-7dda-11e8-80e8-f48c50709a40/roslaunch-alban-GL502VSK-14334.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://127.0.0.1:35217/
ros_comm version 1.12.13

SUMMARY
========

PARAMETERS
* /rosdistro: kinetic
* /rosversion: 1.12.13

NODES

auto-starting new master
process[master]: started with pid [14345]
ROS_MASTER_URI=http://127.0.0.1:11311/

setting /run_id to f9874628-7dda-11e8-80e8-f48c50709a40
process[rosout-1]: started with pid [14358]
started core service [/rosout]
[2] 14375
[ INFO] [1530524008.748696514]: Waiting For connections on 0.0.0.0:8081
[3] 14404
[INFO] [2018.7.2 11:33:29 /home/alban/Documents/NRP/nest-simulator/nestkernel/rng_manager.cpp:226 @ Network::create_rngs_] : Creating default RNGs
[INFO] [2018.7.2 11:33:29 /home/alban/Documents/NRP/nest-simulator/nestkernel/rng_manager.cpp:221 @ Network::create_rngs_] : Deleting existing random number generators
[INFO] [2018.7.2 11:33:29 /home/alban/Documents/NRP/nest-simulator/nestkernel/rng_manager.cpp:226 @ Network::create_rngs_] : Creating default RNGs
[INFO] [2018.7.2 11:33:29 /home/alban/Documents/NRP/nest-simulator/nestkernel/rng_manager.cpp:272 @ Network::create_grng_] : Creating new default global RNG

-- N E S T --

This program is provided AS IS and comes with
NO WARRANTY. See the file LICENSE for details.

Problems or suggestions?
Visit http://www.nest-simulator.org

-- N E S T --

This program is provided AS IS and comes with
NO WARRANTY. See the file LICENSE for details.

Problems or suggestions?
Visit http://www.nest-simulator.org

Type 'nest.help()' to find out more about NEST.
CSAConnector: libneurosim support not available in NEST.
Falling back on PyNN's default CSAConnector.
Please re-compile NEST using --with-libneurosim=PATH
[5] 14445
2018-07-02 11:33:31,677 [MainThread ] [hbp_nrp_back] [WARNING] Application started with uWSGI or any other framework. logging to console by default !
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x1ebaa50 pid: 14430 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 14430, cores: 8)
registered capabilities (classes):
- rosbridge_library.capabilities.call_service.CallService
- rosbridge_library.capabilities.advertise.Advertise
- rosbridge_library.capabilities.publish.Publish
- rosbridge_library.capabilities.subscribe.Subscribe
- <class 'rosbridge_library.capabilities.defragmentation.Defragment'>
- rosbridge_library.capabilities.advertise_service.AdvertiseService
- rosbridge_library.capabilities.service_response.ServiceResponse
- rosbridge_library.capabilities.unadvertise_service.UnadvertiseService
[INFO] [1530524012.062698, 0.000000]: Rosbridge WebSocket server started on port 9090
2018-07-02 11:33:32 GMT+0200 Polling Backend Servers for Experiments, Health & Running Simulations every 5000 ms.
2018-07-02 11:33:32 GMT+0200 Listening on port: 8443
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/benchmark_p3dx/BenchmarkPioneer.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_husky_distributed_music/ExDDistributedBrainHuskyMUSIC.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_husky_distributed/ExDDistributedBrainHusky.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_husky_holodeck/ExDXMLExample_holodeck.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_husky_sbc/ExDBraitenbergHuskySBC.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_husky_spinnaker/HuskySpinnaker.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_husky/ExDXMLExample.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_lauron/ExDBraitenbergLauron.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_lauron_sbc/ExDBraitenbergLauronSBC.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_mouse_holodeck_lab/ExDBraitenbergMouseHoloLab.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_mouse_lab/ExDBraitenbergMouseLab.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/braitenberg_mouse/ExDBraitenbergMouse.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/cdp1_mouse_holodeck/ExDMouseCDP1_holodeck.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/cdp1_mouse_tactile/ExDMouseCDP1_tactile.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/cdp1_mouse/ExDMouseCDP1.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/CDP4_experiment/cdp4.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/CDP4_WP2/CDP4_WP2.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/Conjoint_Experiment_WP2/Conjoint.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/Crowding/Crowding.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/demo_dvs_icub/ExDDvsIcub.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/demo_husky_sbc/ExDDemoHuskySBC.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/demo_manipulation/ExDDemoManipulation.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/demo_mouse_lab/ExDMouseLabExample.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/docked_mouse/ExDDockedMouse.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/dvs_robot_head/ExDDvsRobotHead.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/Example for Hari/Hari.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/gaze_stabilization_icub/GazeStabilizationIcub.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/hari_icub_cerebellum/ICubCerebellum.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/interactive_manipulation/ExDInteractiveManipulation.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/manipulation_force_holodeck/ExDManipulationForce_holodeck.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/manipulation_force/ExDManipulationForce.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/manipulation_holodeck/ExDManipulation_holodeck.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/manipulation_RL/ExDManipulationRL.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/manipulation/ExDManipulation.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/multiple_robots_tutorial/MultipleRobots.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/myoarm_nst/Myo_NST.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/my_first_experiment/ExDMyFirstExperiment.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/myoarm_small/Myo_small.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/nao/ExDNao.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/neuronal_red_detection_holodeck/NeuronalRedDetection_Husky_holodeck.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/neuronal_red_detection/NeuronalRedDetection_Husky.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/retina_icub/ExDRetinaICubTrackingRG.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/retina_mockup/ExDRetinaICubMockup.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/roboy/roboy.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/screen_switching_holodeck/ScreenSwitchingHuskyExperiment_holodeck.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/screen_switching/ScreenSwitchingHuskyExperiment.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/spinnaker_async/ExDSpiNNakerAsynchronous.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/spinnaker_example/ExDSpiNNakerExample.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/spinnaker_mvp_I/spinnaker_mvp_i.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/spinnaker_mvp_II/spinnaker_mvp_ii.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_husky_timeout/ExDTemplateHuskyTimeout.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_husky/ExDTemplateHusky.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_icub_holodeck/ExDTemplateICubHolodeck.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_icub/ExDTemplateICub.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_lauron/ExDTemplateLauron.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_lauron_docked/ExDTemplateDockedLauron.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_manipulation/ExDTemplateManipulation.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_mmm/ExDTemplateMMM.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_mouse/ExDTemplateMouseV2.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_pioneer_feature/ExDTemplatePioneer.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_new/TemplateNew.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/template_pioneer/ExDTemplatePioneer.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/tigrillo_CPG/ExDTigrilloCPG.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/tigrillo_real/TigrilloReal.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/tutorial_baseball_exercise/ExDTutorialBaseballExercise.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/tutorial_baseball_solution/ExDTutorialBaseballSolution.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/tutorial_tensorflow_husky/TensorFlowTutorial.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/user-avatar_test-environment/user-avatar_test-env.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/visual_tracking_icub_holodeck/ExDVisualTrackingICub_holodeck.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/visual_tracking_icub/ExDVisualTrackingICub.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/wp2_featuring/Conjoint.exc
2018-07-02 11:33:32 GMT+0200 Parsing experiment file /home/alban/Documents/NRP/Experiments/zip_robot/ExDXMLExampleRobotZip.exc
2018-07-02 11:33:39 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /identity/me
2018-07-02 11:33:39 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /identity/me
2018-07-02 11:33:39 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/experiments
2018-07-02 11:33:39 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /availableServers
2018-07-02 11:33:39 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/wp2_featuring_0
2018-07-02 11:33:39 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/CDP4_experiment_0
2018-07-02 11:33:39 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/experiment_configuration.exc?byname=true
2018-07-02 11:33:39 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/experiment_configuration.exc?byname=true
2018-07-02 11:33:39 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/Conjoint.jpg?byname=true
2018-07-02 11:33:39 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/cdp4.png?byname=true
2018-07-02 11:33:43 GMT+0200 [REQUEST from ::ffff:127.0.0.1] DELETE /storage/CDP4_experiment_0
2018-07-02 11:33:43 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/experiments
2018-07-02 11:33:43 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /availableServers
2018-07-02 11:33:43 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/wp2_featuring_0
2018-07-02 11:33:43 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/experiment_configuration.exc?byname=true
2018-07-02 11:33:43 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/Conjoint.jpg?byname=true
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /availableServers
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experiments
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/BenchmarkPioneer
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDMouseCDP1
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/cdp4
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/CDP4_WP2
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/Crowding
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDManipulationForce
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/Hari
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDDemoManipulation
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDManipulation
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDMouseCDP1_holodeck
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDManipulationRL
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDXMLExample_holodeck
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ScreenSwitchingHuskyExperiment_holodeck
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDVisualTrackingICub_holodeck
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDBraitenbergMouseHoloLab
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDTemplateICubHolodeck
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDBraitenbergHuskySBC
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDDemoHuskySBC
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/NeuronalRedDetection_Husky
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDBraitenbergMouse
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDBraitenbergMouseLab
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDMouseLabExample
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDMyFirstExperiment
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDTemplateManipulation
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDTemplateHusky
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDTemplateICub
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDTemplatePioneer
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDTutorialBaseballExercise
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/ExDTutorialBaseballSolution
2018-07-02 11:33:45 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /experimentImage/Conjoint
2018-07-02 11:33:47 GMT+0200 [REQUEST from ::ffff:127.0.0.1] POST /storage/clone
2018-07-02 11:33:47 GMT+0200 Flattening experiment
2018-07-02 11:33:47 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/experiments
2018-07-02 11:33:47 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /availableServers
2018-07-02 11:33:47 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/wp2_featuring_0
2018-07-02 11:33:47 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/CDP4_experiment_0
2018-07-02 11:33:47 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/experiment_configuration.exc?byname=true
2018-07-02 11:33:47 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/experiment_configuration.exc?byname=true
2018-07-02 11:33:48 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/Conjoint.jpg?byname=true
2018-07-02 11:33:48 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/cdp4.png?byname=true
2018-07-02 11:33:53 GMT+0200 [REQUEST from ::ffff:127.0.0.1] POST /activity_log/simulation_start
2018-07-02 11:33:57 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/experiments
2018-07-02 11:33:58 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /availableServers
2018-07-02 11:33:58 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/wp2_featuring_0
2018-07-02 11:33:58 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/CDP4_experiment_0
2018-07-02 11:33:58 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/experiment_configuration.exc?byname=true
2018-07-02 11:33:58 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/experiment_configuration.exc?byname=true
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /identity/me
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /identity/me/groups
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /identity/me
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/experiments
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /identity/me
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /availableServers
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/wp2_featuring_0
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/CDP4_experiment_0
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/experiment_configuration.exc?byname=true
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/experiment_configuration.exc?byname=true
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/Conjoint.jpg?byname=true
2018-07-02 11:33:59 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/cdp4.png?byname=true
2018-07-02 11:34:03 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /identity/me
2018-07-02 11:34:03 GMT+0200 [REQUEST from ::ffff:127.0.0.1] POST /activity_log/simulation_start
2018-07-02 11:34:03 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /server/localhost
2018-07-02 11:34:03 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0
2018-07-02 11:34:03 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0
2018-07-02 11:34:03 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/bibi_configuration.bibi?byname=true
2018-07-02 11:34:03 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/cdp4.3ds?byname=true
2018-07-02 11:34:03 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/cdp4.png?byname=true
2018-07-02 11:34:04 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/experiment_configuration.exc?byname=true
2018-07-02 11:34:04 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/hollie.sdf?byname=true
2018-07-02 11:34:04 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/idle_brain.py?byname=true
2018-07-02 11:34:04 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/image_to_saliency.py?byname=true
2018-07-02 11:34:04 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/nrp.launch?byname=true
2018-07-02 11:34:04 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/saccade_characteristics_machine.exd?byname=true
2018-07-02 11:34:04 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/world_indoor.sdf?byname=true
2018-07-02 11:34:04,292 [MainThread ] [SaccadeChara] [WARNING] Could not write to specified logfile or no logfile specified, logging to stdout now!
2018-07-02 11:34:04,313 [MainThread ] [SaccadeChara] [INFO] State machine services running for 'SaccadeCharacteristics'
2018-07-02 11:34:04,376 [uWSGIWorker1] [hbp_nrp_back] [INFO] Requesting simulation resources
2018-07-02 11:34:04,377 [uWSGIWorker1] [hbp_nrp_back] [INFO] simulation timeout initialized
2018-07-02 11:34:04,386 [Thread-7 ] [hbp_nrp_cles] [INFO] Create new simulation request
2018-07-02 11:34:04,386 [Thread-7 ] [hbp_nrp_cles] [INFO] No simulation running, starting a new simulation.
2018-07-02 11:34:04,388 [Thread-7 ] [hbp_nrp_cles] [INFO] Preparing new simulation with environment file: /tmp/nrp-simulation-dir/world_indoor.sdf and ExD config file: /tmp/nrp-simulation-dir/experiment_configuration.exc.
2018-07-02 11:34:04,388 [Thread-7 ] [hbp_nrp_cles] [INFO] Starting the experiment closed loop engine.
2018-07-02 11:34:04,389 [Thread-7 ] [hbp_nrp_cles] [INFO] Read XML Files
2018-07-02 11:34:04,417 [Thread-7 ] [hbp_nrp_cles] [INFO] Bibi: bibi_configuration.bibi
2018-07-02 11:34:04,417 [Thread-7 ] [hbp_nrp_cles] [INFO] BibiAbs:/tmp/nrp-simulation-dir/bibi_configuration.bibi
2018-07-02 11:34:04,421 [Thread-7 ] [hbp_nrp_cles] [INFO] Path is /tmp/nrp-simulation-dir/experiment_configuration.exc
2018-07-02 11:34:04,421 [Thread-7 ] [hbp_nrp_cles] [INFO] Setting up backend Notificator
2018-07-02 11:34:04,423 [Thread-7 ] [hbp_nrp_cles] [INFO] ROS notificator initialized
2018-07-02 11:34:04,423 [Thread-7 ] [hbp_nrp_cles] [INFO] Creating CLE Server
2018-07-02 11:34:04,423 [Thread-7 ] [hbp_nrp_cles] [WARNING] No RNG seed specified, generating a random value.
2018-07-02 11:34:04,424 [Thread-7 ] [hbp_nrp_cles] [INFO] RNG seed = 8247579108591365443
2018-07-02 11:34:04,529 [Thread-3 ] [rospy.intern] [INFO] topic[/rosout] adding connection to [/rosout], count 0
2018-07-02 11:34:04,552 [Thread-7 ] [hbp_nrp_cles] [INFO] Looking for physicsEngine tag value in ExDConfig
2018-07-02 11:34:04,552 [Thread-7 ] [hbp_nrp_cles] [INFO] No physics engine specified explicitly. Using default setting 'ode'
... logging to /home/alban/.ros/log/f9874628-7dda-11e8-80e8-f48c50709a40/roslaunch-alban-GL502VSK-14562.log
started roslaunch server http://127.0.0.1:41511/

SUMMARY
========

PARAMETERS
* /head_manager/move_eyes: True
* /head_manager/move_head: True
* /head_manager/probe: False
* /head_manager/recognize: False
* /head_manager/saliency_height: 192
* /head_manager/saliency_width: 256
* /head_manager/shift: False
* /hollie/eye_tilt_position_controller/joint: hollie_eyes_tilt_...
* /hollie/eye_tilt_position_controller/pid/d: 50.0
* /hollie/eye_tilt_position_controller/pid/i: 5.0
* /hollie/eye_tilt_position_controller/pid/p: 300.0
* /hollie/eye_tilt_position_controller/type: position_controll...
* /hollie/joint_position_controller/joints: ['hollie_eyes_til...
* /hollie/joint_position_controller/type: position_controll...
* /hollie/joint_state_controller/publish_rate: 50
* /hollie/joint_state_controller/type: joint_state_contr...
* /hollie/left_eye_pan_position_controller/joint: hollie_left_eye_p...
* /hollie/left_eye_pan_position_controller/pid/d: 50.0
* /hollie/left_eye_pan_position_controller/pid/i: 5.0
* /hollie/left_eye_pan_position_controller/pid/p: 300.0
* /hollie/left_eye_pan_position_controller/type: position_controll...
* /hollie/neck_pitch_position_controller/joint: hollie_neck_pitch...
* /hollie/neck_pitch_position_controller/pid/d: 50.0
* /hollie/neck_pitch_position_controller/pid/i: 5.0
* /hollie/neck_pitch_position_controller/pid/p: 300.0
* /hollie/neck_pitch_position_controller/type: position_controll...
* /hollie/neck_yaw_position_controller/joint: hollie_neck_yaw_j...
* /hollie/neck_yaw_position_controller/pid/d: 50.0
* /hollie/neck_yaw_position_controller/pid/i: 5.0
* /hollie/neck_yaw_position_controller/pid/p: 300.0
* /hollie/neck_yaw_position_controller/type: position_controll...
* /hollie/right_eye_pan_position_controller/joint: hollie_right_eye_...
* /hollie/right_eye_pan_position_controller/pid/d: 50.0
* /hollie/right_eye_pan_position_controller/pid/i: 5.0
* /hollie/right_eye_pan_position_controller/pid/p: 300.0
* /hollie/right_eye_pan_position_controller/type: position_controll...
* /hook:
* /network_input_height: 192
* /network_input_width: 256
* /recognize/graph_path: /home/alban/.opt/...
* /recognize/model_path: /home/alban/.opt/...
* /robot_description: <?xml version="1....
* /robot_state_publisher/publish_frequency: 1000.0
* /robot_state_publisher/use_tf_static: False
* /rosdistro: kinetic
* /rosversion: 1.12.13
* /saliency_file: /home/alban/Docum...
* /saver/filename_format: cdp4/panorama.png
* /saver/save_all_image: False
* /tensorflow_path: /home/alban/.opt/...

NODES
/hollie/
controller_spawner (controller_manager/spawner)
/
head_manager (embodied_attention/head_manager.py)
recognize (embodied_attention/recognize.py)
recorder (rosbag/record)
robot_state_publisher (robot_state_publisher/robot_state_publisher)
saver (image_view/image_saver)
visual_memory_module (ros_holographic/visual_memory_module.py)
visualizer (embodied_attention/visualizer.py)

ROS_MASTER_URI=http://localhost:11311
process[hollie/controller_spawner-1]: started with pid [14580]
process[robot_state_publisher-2]: started with pid [14581]
process[recorder-3]: started with pid [14582]
process[head_manager-4]: started with pid [14591]
[ INFO] [1530524044.895702689]: Subscribing to /clock
[ INFO] [1530524044.897683029]: Subscribing to /fov
process[recognize-5]: started with pid [14614]
[ INFO] [1530524044.900696136]: Subscribing to /hollie/camera/left/camera_info
[ INFO] [1530524044.903972588]: Subscribing to /hollie/eye_tilt_position_controller/command
[ INFO] [1530524044.906490169]: Subscribing to /hollie/left_eye_pan_position_controller/command
process[visual_memory_module-6]: started with pid [14636]
[ INFO] [1530524044.911794260]: Subscribing to /hollie/neck_pitch_position_controller/command
[ INFO] [1530524044.918893932]: Subscribing to /hollie/neck_yaw_position_controller/command
process[visualizer-7]: started with pid [14661]
[ INFO] [1530524044.933655293]: Subscribing to /hollie/right_eye_pan_position_controller/command
process[saver-8]: started with pid [14673]
[ INFO] [1530524044.944188195]: Subscribing to /joint_states
[ INFO] [1530524044.948198197]: Subscribing to /label
[ INFO] [1530524044.959145518]: Subscribing to /pan
[ INFO] [1530524044.962812332]: Subscribing to /probe_results
[ INFO] [1530524044.966646687]: Subscribing to /roi
2018-07-02 11:34:04,966 [Thread-7 ] [hbp_nrp_cles] [INFO] gzserver arguments: --seed 8247579108591365443 -e ode /tmp/nrp-simulation-dir/world_indoor.sdf
2018-07-02 11:34:04,966 [Thread-7 ] [hbp_nrp_cle.] [INFO] Starting gzserver
[ INFO] [1530524044.982592990]: Subscribing to /rosout
[ INFO] [1530524045.003692442]: Subscribing to /saccade_potential_target
[ INFO] [1530524045.016997145]: Subscribing to /saccade_target
[ INFO] [1530524045.022882474]: Subscribing to /status
[ INFO] [1530524045.027590642]: Subscribing to /tilt
[ INFO] [1530524045.040398453]: Subscribing to command
2018-07-02 11:34:05,156 [Thread-3 ] [rospy.intern] [INFO] topic[/rosout] adding connection to [/recorder], count 1
2018-07-02 11:34:05,157 [Thread-3 ] [rospy.intern] [INFO] topic[/rosout] adding connection to [/recorder], count 1

[INFO] [1530524045.187563, 0.000000]: Controller Spawner: Waiting for service controller_manager/load_controller
-------------------------------------
|Holographic Reduced Representations|
-------------------------------------
[INFO] [1530524045.586354, 0.000000]: Client connected. 1 clients total.
[INFO] [1530524045.791034, 0.000000]: HRR Memory Node is ready...
[INFO] [1530524045.984944, 0.000000]: [Client 0] Subscribed to /ros_cle_simulation/status
2018-07-02 11:34:05,995 [Thread-3 ] [rospy.intern] [INFO] topic[/ros_cle_simulation/status] adding connection to [/rosbridge_websocket], count 0
Found /home/alban/.local/bin/gzserver
Starting gzserver: OK.

[ WARN] [1530524047.049618665]: /use_sim_time set to true and no clock published. Still waiting for valid time...
[ INFO] [1530524047.362408529]: Finished loading Gazebo ROS API Plugin.
[ INFO] [1530524047.362637454]: waitForService: Service [/gazebo/set_physics_properties] has not been advertised, waiting...
[ INFO] [1530524047.375979419]: Gazebo ROS Recording Plugin initialized.
[ INFO] [1530524047.381375808]: Gazebo ROS Playback Plugin initialized.
2018-07-02 11:34:08,596 [Thread-14 ] [rospy.intern] [INFO] topic[/clock] adding connection to [http://127.0.0.1:37719/], count 0
2018-07-02 11:34:08,796 [Thread-7 ] [hbp_nrp_cle.] [INFO] Restarting Gazebo web client communication adapter
2018-07-02 11:34:09 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/experiments
2018-07-02 11:34:09 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /availableServers
2018-07-02 11:34:09 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/wp2_featuring_0
2018-07-02 11:34:09 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/CDP4_experiment_0
2018-07-02 11:34:09 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/experiment_configuration.exc?byname=true
2018-07-02 11:34:09 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/experiment_configuration.exc?byname=true
Now using node v0.10.48 (npm v2.15.1)
2018-07-02 11:34:11.014513: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-07-02 11:34:11.108949: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-07-02 11:34:11.109649: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.645
pciBusID: 0000:01:00.0
totalMemory: 7.93GiB freeMemory: 7.59GiB
2018-07-02 11:34:11.109683: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-07-02 11:34:11.324549: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7331 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
Found /home/alban/Documents/NRP/gzweb/gzbridge/ws_server.js
Starting node: OK.

/home/alban/Documents/NRP/CLE/hbp_nrp_cle/hbp_nrp_cle/robotsim/GazeboHelper.py:119: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
2018-07-02 11:34:11,843 [Thread-7 ] [hbp_nrp_cles] [INFO] RobotAbs: /tmp/nrp-simulation-dir/hollie.sdf
Pose message filter parameters:
minimum seconds between successive messages: 0.02
minimum XYZ distance squared between successive messages: 0.00001
minimum Quaternion distance squared between successive messages: 0.0001
Mon Jul 02 2018 11:34:11 GMT+0200 (CEST) Server is listening on port 7681
[ INFO] [1530524052.282033853]: Camera Plugin (robotNamespace = ), Info: the 'robotNamespace' param did not exit
[ INFO] [1530524052.285858891]: Camera Plugin (ns = ) <tf_prefix_>, set to ""
[ INFO] [1530524052.286094616]: Camera Plugin (robotNamespace = ), Info: the 'robotNamespace' param did not exit
[ INFO] [1530524052.288751442]: Camera Plugin (ns = ) <tf_prefix_>, set to ""
2018-07-02 11:34:12,327 [Thread-7 ] [hbp_nrp_cles] [INFO] Preparing CLE Server
2018-07-02 11:34:12,337 [Thread-7 ] [hbp_nrp_cle.] [INFO] Robot control adapter initialized
2018-07-02 11:34:12,359 [Thread-7 ] [hbp_nrp_cle.] [INFO] neuronal simulator initialized
2018-07-02 11:34:12,359 [Thread-7 ] [BrainLoader ] [INFO] Loading brain model from python: /tmp/nrp-simulation-dir/idle_brain.py
2018-07-02 11:34:12,378 [Thread-7 ] [hbp_nrp_cle.] [INFO] Saving brain source
2018-07-02 11:34:12,378 [Thread-7 ] [hbp_nrp_cle.] [INFO] Initialize transfer functions node tfnode
2018-07-02 11:34:12,378 [Thread-7 ] [hbp_nrp_cle.] [INFO] PyNN communication adapter initialized
2018-07-02 11:34:12,379 [Thread-7 ] [hbp_nrp_cle.] [WARNING] ROS node already initialized with another name
2018-07-02 11:34:12,384 [Thread-7 ] [hbp_nrp_cles] [INFO] Registering ROS Service handlers
2018-07-02 11:34:12,386 [Thread-7 ] [hbp_nrp_cles] [INFO] Registering ROS Service handlers
2018-07-02 11:34:12,963 [Thread-3 ] [rospy.intern] [INFO] topic[/ros_cle_simulation/0/lifecycle] adding connection to [/nrp_backend], count 0
2018-07-02 11:34:13.280555: I tensorflow/core/platform/cpu_feature_guard.cc:140]
2018-07-02 11:34:13,282 [Thread-18 ] [rospy.intern] [INFO] topic[/clock] adding connection to [http://127.0.0.1:37719/], count 0
2018-07-02 11:34:13,282 [Thread-17 ] [rospy.intern] [INFO] topic[/ros_cle_simulation/0/lifecycle] adding connection to [http://127.0.0.1:35463/], count 0
2018-07-02 11:34:13,282 [Thread-15 ] [rospy.intern] [INFO] topic[/ros_cle_simulation/0/lifecycle] adding connection to [http://127.0.0.1:36595/], count 0
2018-07-02 11:34:13,284 [Thread-3 ] [rospy.intern] [INFO] topic[/ros_cle_simulation/0/lifecycle] adding connection to [/ros_cle_simulation_14404_1530524010422], count 1
2018-07-02 11:34:13.358846: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-07-02 11:34:13.359468: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.645
pciBusID: 0000:01:00.0
totalMemory: 7.93GiB freeMemory: 275.06MiB
2018-07-02 11:34:13.359506: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2018-07-02 11:34:13.621509: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 219 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
2018-07-02 11:34:13.622443: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 219.31M (229965824 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
[ INFO] [1530524053.917363438]: sdf name plugin, sdf description A plugin is a dynamically loaded chunk of code. It can exist as a child of world, model, and sensor.
[ WARN] [1530524053.917703806]: No controller for joint hollie_eyes_tilt_joint found
Error [Element.cc:684] Missing element description for [visual]
[ WARN] [1530524053.917930171]: No controller for joint hollie_left_eye_pan_joint found
Error [Element.cc:684] Missing element description for [visual]
[ INFO] [1530524053.917999097]: Found controller for joint hollie_neck_pitch_joint
[ INFO] [1530524053.918177757]: Controller PID values p=10000.000000, i=0.000000, d=0.000000
[ INFO] [1530524053.918231927]: Controller has type position
[ INFO] [1530524053.922228943]: Added new position controller for joint hollie_neck_pitch_joint
Error [Element.cc:684] Missing element description for [visual]
[ INFO] [1530524053.922342810]: Found controller for joint hollie_neck_yaw_joint
[ INFO] [1530524053.922376072]: Controller PID values p=10000.000000, i=0.000000, d=0.000000
[ INFO] [1530524053.922410483]: Controller has type position
[ INFO] [1530524053.924965542]: Added new position controller for joint hollie_neck_yaw_joint
Error [Element.cc:684] Missing element description for [visual]
[ WARN] [1530524053.925165402]: No controller for joint hollie_right_eye_pan_joint found
Error [Element.cc:684] Missing element description for [visual]
[ INFO] [1530524053.940658630]: Loading gazebo_ros_control plugin
[ INFO] [1530524053.940844810]: Starting gazebo_ros_control plugin in namespace: /hollie
[ INFO] [1530524053.941533130]: gazebo_ros_control plugin is waiting for model URDF in parameter [/robot_description] on the ROS param server.
[ INFO] [1530524054.081034838]: Loaded gazebo_ros_control.
[ INFO] [1530524054.081395039]: sdf name plugin, sdf description A plugin is a dynamically loaded chunk of code. It can exist as a child of world, model, and sensor.
[ INFO] [1530524054.081428248]: Found controller for joint hollie_eyes_tilt_joint
[ INFO] [1530524054.081478492]: Controller PID values p=10000.000000, i=0.000000, d=0.000000
[ INFO] [1530524054.081531057]: Controller has type position
[ INFO] [1530524054.084498170]: Added new position controller for joint hollie_eyes_tilt_joint
Error [Element.cc:684] Missing element description for [visual]
[ INFO] [1530524054.084627442]: Found controller for joint hollie_left_eye_pan_joint
[ INFO] [1530524054.084666944]: Controller PID values p=10000.000000, i=0.000000, d=0.000000
[ INFO] [1530524054.084700933]: Controller has type position
[ INFO] [1530524054.087813805]: Added new position controller for joint hollie_left_eye_pan_joint
Error [Element.cc:684] Missing element description for [visual]
[ WARN] [1530524054.087899491]: No controller for joint hollie_neck_pitch_joint found
Error [Element.cc:684] Missing element description for [visual]
[ WARN] [1530524054.087956772]: No controller for joint hollie_neck_yaw_joint found
Error [Element.cc:684] Missing element description for [visual]
[ INFO] [1530524054.088006238]: Found controller for joint hollie_right_eye_pan_joint
[ INFO] [1530524054.088038613]: Controller PID values p=10000.000000, i=0.000000, d=0.000000
[ INFO] [1530524054.088063332]: Controller has type position
[ INFO] [1530524054.090551998]: Added new position controller for joint hollie_right_eye_pan_joint
Error [Element.cc:684] Missing element description for [visual]
[ERROR] [1530524054.090741324]: Tried to advertise a service that is already advertised in this node [/robot/set_pid_parameters]
[ERROR] [1530524054.090858032]: Tried to advertise a service that is already advertised in this node [/robot/joint_properties]
[INFO] [1530524054.231543, 0.000000]: Controller Spawner: Waiting for service controller_manager/switch_controller
[INFO] [1530524054.232736, 0.000000]: Controller Spawner: Waiting for service controller_manager/unload_controller
[INFO] [1530524054.233864, 0.000000]: Loading controller: joint_state_controller
[INFO] [1530524054.240262, 0.000000]: Loading controller: eye_tilt_position_controller
[INFO] [1530524054.248415, 0.000000]: Loading controller: left_eye_pan_position_controller
[INFO] [1530524054.254129, 0.000000]: Loading controller: right_eye_pan_position_controller
[INFO] [1530524054.259258, 0.000000]: Loading controller: neck_pitch_position_controller
[INFO] [1530524054.264843, 0.000000]: Loading controller: neck_yaw_position_controller
[INFO] [1530524054.270362, 0.000000]: Controller Spawner: Loaded controllers: joint_state_controller, eye_tilt_position_controller, left_eye_pan_position_controller, right_eye_pan_position_controller, neck_pitch_position_controller, neck_yaw_position_controller
2018-07-02 11:34:14,558 [Thread-7 ] [tensorflow ] [INFO] Restoring parameters from /home/alban/Documents/NRP/GazeboRosPackages/src/embodied_attention/model/model.ckpt
2018-07-02 11:34:14.749946: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.750473: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.751732: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.753199: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.754779: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.755993: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.756894: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.757721: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.758267: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.759044: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.759665: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.760175: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.760982: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.761764: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.762333: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.762895: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.765081: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:14.765638: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:19 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/experiments
2018-07-02 11:34:19 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /availableServers
2018-07-02 11:34:19 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/wp2_featuring_0
2018-07-02 11:34:19 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /joinableServers/CDP4_experiment_0
2018-07-02 11:34:19 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/wp2_featuring_0/experiment_configuration.exc?byname=true
2018-07-02 11:34:19 GMT+0200 [REQUEST from ::ffff:127.0.0.1] GET /storage/CDP4_experiment_0/experiment_configuration.exc?byname=true
2018-07-02 11:34:24.752171: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.753884: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.753950: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.25MiB. Current allocation summary follows.
2018-07-02 11:34:24.753986: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (256): Total Chunks: 9, Chunks in use: 8. 2.2KiB allocated for chunks. 2.0KiB in use in bin. 1.5KiB client-requested in use in bin.
2018-07-02 11:34:24.754008: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (512): Total Chunks: 7, Chunks in use: 6. 3.8KiB allocated for chunks. 3.2KiB in use in bin. 3.0KiB client-requested in use in bin.
2018-07-02 11:34:24.754027: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1024): Total Chunks: 7, Chunks in use: 7. 7.2KiB allocated for chunks. 7.2KiB in use in bin. 7.0KiB client-requested in use in bin.
2018-07-02 11:34:24.754048: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2048): Total Chunks: 24, Chunks in use: 24. 48.8KiB allocated for chunks. 48.8KiB in use in bin. 48.0KiB client-requested in use in bin.
2018-07-02 11:34:24.754069: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4096): Total Chunks: 3, Chunks in use: 3. 20.2KiB allocated for chunks. 20.2KiB in use in bin. 20.2KiB client-requested in use in bin.
2018-07-02 11:34:24.754092: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.754115: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.754138: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.754167: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (65536): Total Chunks: 1, Chunks in use: 0. 124.0KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.754199: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (131072): Total Chunks: 3, Chunks in use: 3. 432.0KiB allocated for chunks. 432.0KiB in use in bin. 432.0KiB client-requested in use in bin.
2018-07-02 11:34:24.754237: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (262144): Total Chunks: 4, Chunks in use: 3. 1.12MiB allocated for chunks. 864.0KiB in use in bin. 864.0KiB client-requested in use in bin.
2018-07-02 11:34:24.754271: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (524288): Total Chunks: 3, Chunks in use: 2. 1.69MiB allocated for chunks. 1.12MiB in use in bin. 1.12MiB client-requested in use in bin.
2018-07-02 11:34:24.754303: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1048576): Total Chunks: 1, Chunks in use: 1. 1.09MiB allocated for chunks. 1.09MiB in use in bin. 576.0KiB client-requested in use in bin.
2018-07-02 11:34:24.754336: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2097152): Total Chunks: 3, Chunks in use: 3. 6.75MiB allocated for chunks. 6.75MiB in use in bin. 6.75MiB client-requested in use in bin.
2018-07-02 11:34:24.754451: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4194304): Total Chunks: 2, Chunks in use: 2. 9.00MiB allocated for chunks. 9.00MiB in use in bin. 9.00MiB client-requested in use in bin.
2018-07-02 11:34:24.754491: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8388608): Total Chunks: 19, Chunks in use: 19. 177.11MiB allocated for chunks. 177.11MiB in use in bin. 166.50MiB client-requested in use in bin.
2018-07-02 11:34:24.754521: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16777216): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.754547: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (33554432): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.754573: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.754598: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.754621: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.754638: I tensorflow/core/common_runtime/bfc_allocator.cc:646] Bin for 2.25MiB was 2.00MiB, Chunk State:
2018-07-02 11:34:24.754654: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00000 of size 1280
2018-07-02 11:34:24.754666: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00500 of size 256
2018-07-02 11:34:24.754677: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00600 of size 256
2018-07-02 11:34:24.754688: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00700 of size 256
2018-07-02 11:34:24.754698: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00800 of size 256
2018-07-02 11:34:24.754708: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00900 of size 256
2018-07-02 11:34:24.754719: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00a00 of size 256
2018-07-02 11:34:24.754732: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1cfa00b00 of size 256
2018-07-02 11:34:24.754748: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00c00 of size 256
2018-07-02 11:34:24.754768: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00d00 of size 2048
2018-07-02 11:34:24.754786: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01500 of size 2048
2018-07-02 11:34:24.754804: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01d00 of size 2048
2018-07-02 11:34:24.754822: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02500 of size 2048
2018-07-02 11:34:24.754840: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02d00 of size 9426944
2018-07-02 11:34:24.754861: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0300500 of size 9439232
2018-07-02 11:34:24.754880: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c00d00 of size 2048
2018-07-02 11:34:24.754899: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01500 of size 2048
2018-07-02 11:34:24.754918: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01d00 of size 4718592
2018-07-02 11:34:24.754937: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1081d00 of size 2048
2018-07-02 11:34:24.754956: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082500 of size 2048
2018-07-02 11:34:24.754973: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082d00 of size 256
2018-07-02 11:34:24.754993: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1082e00 of size 512
2018-07-02 11:34:24.755012: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083000 of size 512
2018-07-02 11:34:24.755030: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083200 of size 768
2018-07-02 11:34:24.755048: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083500 of size 2048
2018-07-02 11:34:24.755067: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083d00 of size 2048
2018-07-02 11:34:24.755080: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1084500 of size 6912
2018-07-02 11:34:24.755091: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086000 of size 512
2018-07-02 11:34:24.755101: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086200 of size 512
2018-07-02 11:34:24.755112: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086400 of size 512
2018-07-02 11:34:24.755123: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086600 of size 1024
2018-07-02 11:34:24.755134: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086a00 of size 1024
2018-07-02 11:34:24.755144: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086e00 of size 512
2018-07-02 11:34:24.755155: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087000 of size 2816
2018-07-02 11:34:24.755166: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087b00 of size 6912
2018-07-02 11:34:24.755177: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1089600 of size 6912
2018-07-02 11:34:24.755187: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b100 of size 2048
2018-07-02 11:34:24.755198: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b900 of size 2048
2018-07-02 11:34:24.755208: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c100 of size 2048
2018-07-02 11:34:24.755219: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c900 of size 2048
2018-07-02 11:34:24.755235: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d100 of size 2048
2018-07-02 11:34:24.755254: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d900 of size 2048
2018-07-02 11:34:24.755271: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e100 of size 2048
2018-07-02 11:34:24.755282: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e900 of size 1024
2018-07-02 11:34:24.755292: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108ed00 of size 2048
2018-07-02 11:34:24.755303: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f500 of size 1024
2018-07-02 11:34:24.755313: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f900 of size 1024
2018-07-02 11:34:24.755324: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108fd00 of size 1024
2018-07-02 11:34:24.755334: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1090100 of size 126976
2018-07-02 11:34:24.755351: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10af100 of size 147456
2018-07-02 11:34:24.755368: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10d3100 of size 147456
2018-07-02 11:34:24.755386: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d10f7100 of size 294912
2018-07-02 11:34:24.755406: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d113f100 of size 147456
2018-07-02 11:34:24.755427: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1163100 of size 294912
2018-07-02 11:34:24.755444: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11ab100 of size 294912
2018-07-02 11:34:24.755456: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11f3100 of size 589824
2018-07-02 11:34:24.755467: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1283100 of size 589824
2018-07-02 11:34:24.755481: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1313100 of size 589824
2018-07-02 11:34:24.755498: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13a3100 of size 294912
2018-07-02 11:34:24.755517: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13eb100 of size 1141760
2018-07-02 11:34:24.755536: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1501d00 of size 9437184
2018-07-02 11:34:24.755555: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1e01d00 of size 9437184
2018-07-02 11:34:24.755569: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2701d00 of size 2048
2018-07-02 11:34:24.755580: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702500 of size 2048
2018-07-02 11:34:24.755591: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702d00 of size 2048
2018-07-02 11:34:24.755602: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703500 of size 2048
2018-07-02 11:34:24.755613: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703d00 of size 9437184
2018-07-02 11:34:24.755624: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3003d00 of size 9437184
2018-07-02 11:34:24.755635: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3903d00 of size 9437184
2018-07-02 11:34:24.755646: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4203d00 of size 9437184
2018-07-02 11:34:24.755656: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4b03d00 of size 9437184
2018-07-02 11:34:24.755667: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5403d00 of size 2048
2018-07-02 11:34:24.755677: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5404500 of size 4718592
2018-07-02 11:34:24.755689: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5884500 of size 2359296
2018-07-02 11:34:24.755699: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5ac4500 of size 2359296
2018-07-02 11:34:24.755710: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5d04500 of size 9437184
2018-07-02 11:34:24.755721: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6604500 of size 9437184
2018-07-02 11:34:24.755733: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6f04500 of size 9437184
2018-07-02 11:34:24.755744: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d7804500 of size 9437184
2018-07-02 11:34:24.755755: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8104500 of size 9437184
2018-07-02 11:34:24.755767: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8a04500 of size 9437184
2018-07-02 11:34:24.755784: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9304500 of size 9437184
2018-07-02 11:34:24.755802: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9c04500 of size 9437184
2018-07-02 11:34:24.755820: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da504500 of size 2359296
2018-07-02 11:34:24.755838: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da744500 of size 9437184
2018-07-02 11:34:24.755859: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1db044500 of size 15848704
2018-07-02 11:34:24.755877: I tensorflow/core/common_runtime/bfc_allocator.cc:671] Summary of in-use Chunks by size:
2018-07-02 11:34:24.755905: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 8 Chunks of size 256 totalling 2.0KiB
2018-07-02 11:34:24.755930: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 5 Chunks of size 512 totalling 2.5KiB
2018-07-02 11:34:24.755953: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 768 totalling 768B
2018-07-02 11:34:24.755973: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 6 Chunks of size 1024 totalling 6.0KiB
2018-07-02 11:34:24.755987: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1280 totalling 1.2KiB
2018-07-02 11:34:24.756001: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 23 Chunks of size 2048 totalling 46.0KiB
2018-07-02 11:34:24.756014: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 2816 totalling 2.8KiB
2018-07-02 11:34:24.756028: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 6912 totalling 20.2KiB
2018-07-02 11:34:24.756041: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 147456 totalling 432.0KiB
2018-07-02 11:34:24.756054: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 294912 totalling 864.0KiB
2018-07-02 11:34:24.756067: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 589824 totalling 1.12MiB
2018-07-02 11:34:24.756080: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1141760 totalling 1.09MiB
2018-07-02 11:34:24.756092: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 2359296 totalling 6.75MiB
2018-07-02 11:34:24.756105: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 4718592 totalling 9.00MiB
2018-07-02 11:34:24.756117: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9426944 totalling 8.99MiB
2018-07-02 11:34:24.756130: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 16 Chunks of size 9437184 totalling 144.00MiB
2018-07-02 11:34:24.756143: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9439232 totalling 9.00MiB
2018-07-02 11:34:24.756156: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 15848704 totalling 15.11MiB
2018-07-02 11:34:24.756288: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Sum Total of in-use chunks: 196.42MiB
2018-07-02 11:34:24.756347: I tensorflow/core/common_runtime/bfc_allocator.cc:680] Stats:
Limit: 229965824
InUse: 205956864
MaxInUse: 205956864
NumAllocs: 156
MaxAllocSize: 15848704

2018-07-02 11:34:24.756400: W tensorflow/core/common_runtime/bfc_allocator.cc:279] ***x*********************************************************************************************xxx
2018-07-02 11:34:24.756461: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at assign_op.h:111 : Resource exhausted: OOM when allocating tensor with shape[3,3,256,256] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
2018-07-02 11:34:24.757989: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.759370: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.759410: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 9.00MiB. Current allocation summary follows.
2018-07-02 11:34:24.759440: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (256): Total Chunks: 9, Chunks in use: 8. 2.2KiB allocated for chunks. 2.0KiB in use in bin. 1.5KiB client-requested in use in bin.
2018-07-02 11:34:24.759461: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (512): Total Chunks: 7, Chunks in use: 6. 3.8KiB allocated for chunks. 3.2KiB in use in bin. 3.0KiB client-requested in use in bin.
2018-07-02 11:34:24.759480: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1024): Total Chunks: 7, Chunks in use: 7. 7.2KiB allocated for chunks. 7.2KiB in use in bin. 7.0KiB client-requested in use in bin.
2018-07-02 11:34:24.759500: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2048): Total Chunks: 24, Chunks in use: 24. 48.8KiB allocated for chunks. 48.8KiB in use in bin. 48.0KiB client-requested in use in bin.
2018-07-02 11:34:24.759521: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4096): Total Chunks: 3, Chunks in use: 3. 20.2KiB allocated for chunks. 20.2KiB in use in bin. 20.2KiB client-requested in use in bin.
2018-07-02 11:34:24.759539: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.759557: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.759573: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.759591: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (65536): Total Chunks: 1, Chunks in use: 0. 124.0KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.759612: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (131072): Total Chunks: 3, Chunks in use: 3. 432.0KiB allocated for chunks. 432.0KiB in use in bin. 432.0KiB client-requested in use in bin.
2018-07-02 11:34:24.759632: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (262144): Total Chunks: 4, Chunks in use: 3. 1.12MiB allocated for chunks. 864.0KiB in use in bin. 864.0KiB client-requested in use in bin.
2018-07-02 11:34:24.759650: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (524288): Total Chunks: 3, Chunks in use: 2. 1.69MiB allocated for chunks. 1.12MiB in use in bin. 1.12MiB client-requested in use in bin.
2018-07-02 11:34:24.759668: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1048576): Total Chunks: 1, Chunks in use: 1. 1.09MiB allocated for chunks. 1.09MiB in use in bin. 576.0KiB client-requested in use in bin.
2018-07-02 11:34:24.759686: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2097152): Total Chunks: 3, Chunks in use: 3. 6.75MiB allocated for chunks. 6.75MiB in use in bin. 6.75MiB client-requested in use in bin.
2018-07-02 11:34:24.759704: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4194304): Total Chunks: 2, Chunks in use: 2. 9.00MiB allocated for chunks. 9.00MiB in use in bin. 9.00MiB client-requested in use in bin.
2018-07-02 11:34:24.759724: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8388608): Total Chunks: 19, Chunks in use: 19. 177.11MiB allocated for chunks. 177.11MiB in use in bin. 166.50MiB client-requested in use in bin.
2018-07-02 11:34:24.759741: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16777216): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.759758: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (33554432): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.759777: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.759803: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.759831: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.759861: I tensorflow/core/common_runtime/bfc_allocator.cc:646] Bin for 9.00MiB was 8.00MiB, Chunk State:
2018-07-02 11:34:24.759880: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00000 of size 1280
2018-07-02 11:34:24.759902: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00500 of size 256
2018-07-02 11:34:24.759918: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00600 of size 256
2018-07-02 11:34:24.759930: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00700 of size 256
2018-07-02 11:34:24.759943: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00800 of size 256
2018-07-02 11:34:24.759956: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00900 of size 256
2018-07-02 11:34:24.759968: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00a00 of size 256
2018-07-02 11:34:24.759981: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1cfa00b00 of size 256
2018-07-02 11:34:24.759994: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00c00 of size 256
2018-07-02 11:34:24.760007: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00d00 of size 2048
2018-07-02 11:34:24.760020: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01500 of size 2048
2018-07-02 11:34:24.760033: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01d00 of size 2048
2018-07-02 11:34:24.760045: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02500 of size 2048
2018-07-02 11:34:24.760058: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02d00 of size 9426944
2018-07-02 11:34:24.760072: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0300500 of size 9439232
2018-07-02 11:34:24.760084: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c00d00 of size 2048
2018-07-02 11:34:24.760097: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01500 of size 2048
2018-07-02 11:34:24.760111: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01d00 of size 4718592
2018-07-02 11:34:24.760124: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1081d00 of size 2048
2018-07-02 11:34:24.760137: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082500 of size 2048
2018-07-02 11:34:24.760149: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082d00 of size 256
2018-07-02 11:34:24.762535: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1082e00 of size 512
2018-07-02 11:34:24.762563: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083000 of size 512
2018-07-02 11:34:24.762577: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083200 of size 768
2018-07-02 11:34:24.762590: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083500 of size 2048
2018-07-02 11:34:24.762603: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083d00 of size 2048
2018-07-02 11:34:24.762617: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1084500 of size 6912
2018-07-02 11:34:24.762630: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086000 of size 512
2018-07-02 11:34:24.762643: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086200 of size 512
2018-07-02 11:34:24.762656: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086400 of size 512
2018-07-02 11:34:24.762669: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086600 of size 1024
2018-07-02 11:34:24.762686: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086a00 of size 1024
2018-07-02 11:34:24.762709: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086e00 of size 512
2018-07-02 11:34:24.762734: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087000 of size 2816
2018-07-02 11:34:24.762759: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087b00 of size 6912
2018-07-02 11:34:24.762782: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1089600 of size 6912
2018-07-02 11:34:24.762802: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b100 of size 2048
2018-07-02 11:34:24.762819: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b900 of size 2048
2018-07-02 11:34:24.762832: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c100 of size 2048
2018-07-02 11:34:24.762845: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c900 of size 2048
2018-07-02 11:34:24.762857: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d100 of size 2048
2018-07-02 11:34:24.762870: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d900 of size 2048
2018-07-02 11:34:24.762882: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e100 of size 2048
2018-07-02 11:34:24.762895: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e900 of size 1024
2018-07-02 11:34:24.762907: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108ed00 of size 2048
2018-07-02 11:34:24.762920: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f500 of size 1024
2018-07-02 11:34:24.762932: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f900 of size 1024
2018-07-02 11:34:24.762945: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108fd00 of size 1024
2018-07-02 11:34:24.762958: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1090100 of size 126976
2018-07-02 11:34:24.762978: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10af100 of size 147456
2018-07-02 11:34:24.762991: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10d3100 of size 147456
2018-07-02 11:34:24.763004: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d10f7100 of size 294912
2018-07-02 11:34:24.763017: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d113f100 of size 147456
2018-07-02 11:34:24.763030: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1163100 of size 294912
2018-07-02 11:34:24.763043: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11ab100 of size 294912
2018-07-02 11:34:24.763056: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11f3100 of size 589824
2018-07-02 11:34:24.763069: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1283100 of size 589824
2018-07-02 11:34:24.763082: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1313100 of size 589824
2018-07-02 11:34:24.763095: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13a3100 of size 294912
2018-07-02 11:34:24.763108: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13eb100 of size 1141760
2018-07-02 11:34:24.763121: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1501d00 of size 9437184
2018-07-02 11:34:24.763134: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1e01d00 of size 9437184
2018-07-02 11:34:24.763147: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2701d00 of size 2048
2018-07-02 11:34:24.763159: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702500 of size 2048
2018-07-02 11:34:24.763172: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702d00 of size 2048
2018-07-02 11:34:24.763184: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703500 of size 2048
2018-07-02 11:34:24.763197: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703d00 of size 9437184
2018-07-02 11:34:24.763209: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3003d00 of size 9437184
2018-07-02 11:34:24.763222: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3903d00 of size 9437184
2018-07-02 11:34:24.763235: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4203d00 of size 9437184
2018-07-02 11:34:24.763247: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4b03d00 of size 9437184
2018-07-02 11:34:24.763260: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5403d00 of size 2048
2018-07-02 11:34:24.763273: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5404500 of size 4718592
2018-07-02 11:34:24.763286: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5884500 of size 2359296
2018-07-02 11:34:24.763299: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5ac4500 of size 2359296
2018-07-02 11:34:24.763312: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5d04500 of size 9437184
2018-07-02 11:34:24.763324: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6604500 of size 9437184
2018-07-02 11:34:24.763337: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6f04500 of size 9437184
2018-07-02 11:34:24.763350: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d7804500 of size 9437184
2018-07-02 11:34:24.763362: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8104500 of size 9437184
2018-07-02 11:34:24.763375: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8a04500 of size 9437184
2018-07-02 11:34:24.763392: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9304500 of size 9437184
2018-07-02 11:34:24.763411: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9c04500 of size 9437184
2018-07-02 11:34:24.763429: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da504500 of size 2359296
2018-07-02 11:34:24.763446: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da744500 of size 9437184
2018-07-02 11:34:24.763464: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1db044500 of size 15848704
2018-07-02 11:34:24.763480: I tensorflow/core/common_runtime/bfc_allocator.cc:671] Summary of in-use Chunks by size:
2018-07-02 11:34:24.763504: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 8 Chunks of size 256 totalling 2.0KiB
2018-07-02 11:34:24.763523: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 5 Chunks of size 512 totalling 2.5KiB
2018-07-02 11:34:24.770535: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 768 totalling 768B
2018-07-02 11:34:24.770609: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 6 Chunks of size 1024 totalling 6.0KiB
2018-07-02 11:34:24.770640: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1280 totalling 1.2KiB
2018-07-02 11:34:24.770667: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 23 Chunks of size 2048 totalling 46.0KiB
2018-07-02 11:34:24.770691: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 2816 totalling 2.8KiB
2018-07-02 11:34:24.770718: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 6912 totalling 20.2KiB
2018-07-02 11:34:24.770745: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 147456 totalling 432.0KiB
2018-07-02 11:34:24.770771: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 294912 totalling 864.0KiB
2018-07-02 11:34:24.770798: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 589824 totalling 1.12MiB
2018-07-02 11:34:24.770823: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1141760 totalling 1.09MiB
2018-07-02 11:34:24.770850: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 2359296 totalling 6.75MiB
2018-07-02 11:34:24.770876: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 4718592 totalling 9.00MiB
2018-07-02 11:34:24.770904: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9426944 totalling 8.99MiB
2018-07-02 11:34:24.770931: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 16 Chunks of size 9437184 totalling 144.00MiB
2018-07-02 11:34:24.770958: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9439232 totalling 9.00MiB
2018-07-02 11:34:24.770986: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 15848704 totalling 15.11MiB
2018-07-02 11:34:24.771014: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Sum Total of in-use chunks: 196.42MiB
2018-07-02 11:34:24.771045: I tensorflow/core/common_runtime/bfc_allocator.cc:680] Stats:
Limit: 229965824
InUse: 205956864
MaxInUse: 205956864
NumAllocs: 156
MaxAllocSize: 15848704

2018-07-02 11:34:24.771094: W tensorflow/core/common_runtime/bfc_allocator.cc:279] ***x*********************************************************************************************xxx
2018-07-02 11:34:24.772893: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.774306: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.774349: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 9.00MiB. Current allocation summary follows.
2018-07-02 11:34:24.774390: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (256): Total Chunks: 9, Chunks in use: 8. 2.2KiB allocated for chunks. 2.0KiB in use in bin. 1.5KiB client-requested in use in bin.
2018-07-02 11:34:24.774455: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (512): Total Chunks: 7, Chunks in use: 6. 3.8KiB allocated for chunks. 3.2KiB in use in bin. 3.0KiB client-requested in use in bin.
2018-07-02 11:34:24.774486: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1024): Total Chunks: 7, Chunks in use: 7. 7.2KiB allocated for chunks. 7.2KiB in use in bin. 7.0KiB client-requested in use in bin.
2018-07-02 11:34:24.774519: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2048): Total Chunks: 24, Chunks in use: 24. 48.8KiB allocated for chunks. 48.8KiB in use in bin. 48.0KiB client-requested in use in bin.
2018-07-02 11:34:24.774549: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4096): Total Chunks: 3, Chunks in use: 3. 20.2KiB allocated for chunks. 20.2KiB in use in bin. 20.2KiB client-requested in use in bin.
2018-07-02 11:34:24.774578: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.774601: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.774616: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.774633: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (65536): Total Chunks: 1, Chunks in use: 0. 124.0KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.774652: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (131072): Total Chunks: 3, Chunks in use: 3. 432.0KiB allocated for chunks. 432.0KiB in use in bin. 432.0KiB client-requested in use in bin.
2018-07-02 11:34:24.774669: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (262144): Total Chunks: 4, Chunks in use: 3. 1.12MiB allocated for chunks. 864.0KiB in use in bin. 864.0KiB client-requested in use in bin.
2018-07-02 11:34:24.774686: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (524288): Total Chunks: 3, Chunks in use: 2. 1.69MiB allocated for chunks. 1.12MiB in use in bin. 1.12MiB client-requested in use in bin.
2018-07-02 11:34:24.774702: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1048576): Total Chunks: 1, Chunks in use: 1. 1.09MiB allocated for chunks. 1.09MiB in use in bin. 576.0KiB client-requested in use in bin.
2018-07-02 11:34:24.774721: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2097152): Total Chunks: 3, Chunks in use: 3. 6.75MiB allocated for chunks. 6.75MiB in use in bin. 6.75MiB client-requested in use in bin.
2018-07-02 11:34:24.774747: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4194304): Total Chunks: 2, Chunks in use: 2. 9.00MiB allocated for chunks. 9.00MiB in use in bin. 9.00MiB client-requested in use in bin.
2018-07-02 11:34:24.774780: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8388608): Total Chunks: 19, Chunks in use: 19. 177.11MiB allocated for chunks. 177.11MiB in use in bin. 166.50MiB client-requested in use in bin.
2018-07-02 11:34:24.774810: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16777216): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.774834: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (33554432): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.774861: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.774882: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.774898: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.778551: I tensorflow/core/common_runtime/bfc_allocator.cc:646] Bin for 9.00MiB was 8.00MiB, Chunk State:
2018-07-02 11:34:24.778600: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00000 of size 1280
2018-07-02 11:34:24.778617: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00500 of size 256
2018-07-02 11:34:24.778631: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00600 of size 256
2018-07-02 11:34:24.778645: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00700 of size 256
2018-07-02 11:34:24.778659: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00800 of size 256
2018-07-02 11:34:24.778672: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00900 of size 256
2018-07-02 11:34:24.778685: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00a00 of size 256
2018-07-02 11:34:24.778698: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1cfa00b00 of size 256
2018-07-02 11:34:24.778710: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00c00 of size 256
2018-07-02 11:34:24.778723: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00d00 of size 2048
2018-07-02 11:34:24.778736: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01500 of size 2048
2018-07-02 11:34:24.778749: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01d00 of size 2048
2018-07-02 11:34:24.778761: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02500 of size 2048
2018-07-02 11:34:24.778775: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02d00 of size 9426944
2018-07-02 11:34:24.778788: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0300500 of size 9439232
2018-07-02 11:34:24.778800: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c00d00 of size 2048
2018-07-02 11:34:24.778813: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01500 of size 2048
2018-07-02 11:34:24.778829: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01d00 of size 4718592
2018-07-02 11:34:24.778849: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1081d00 of size 2048
2018-07-02 11:34:24.778869: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082500 of size 2048
2018-07-02 11:34:24.778883: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082d00 of size 256
2018-07-02 11:34:24.778896: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1082e00 of size 512
2018-07-02 11:34:24.778910: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083000 of size 512
2018-07-02 11:34:24.778924: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083200 of size 768
2018-07-02 11:34:24.778937: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083500 of size 2048
2018-07-02 11:34:24.778950: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083d00 of size 2048
2018-07-02 11:34:24.778963: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1084500 of size 6912
2018-07-02 11:34:24.778976: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086000 of size 512
2018-07-02 11:34:24.778988: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086200 of size 512
2018-07-02 11:34:24.779001: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086400 of size 512
2018-07-02 11:34:24.779014: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086600 of size 1024
2018-07-02 11:34:24.779026: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086a00 of size 1024
2018-07-02 11:34:24.779039: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086e00 of size 512
2018-07-02 11:34:24.779052: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087000 of size 2816
2018-07-02 11:34:24.779065: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087b00 of size 6912
2018-07-02 11:34:24.779077: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1089600 of size 6912
2018-07-02 11:34:24.779090: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b100 of size 2048
2018-07-02 11:34:24.779102: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b900 of size 2048
2018-07-02 11:34:24.779115: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c100 of size 2048
2018-07-02 11:34:24.779127: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c900 of size 2048
2018-07-02 11:34:24.779139: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d100 of size 2048
2018-07-02 11:34:24.779152: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d900 of size 2048
2018-07-02 11:34:24.779168: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e100 of size 2048
2018-07-02 11:34:24.779188: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e900 of size 1024
2018-07-02 11:34:24.779206: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108ed00 of size 2048
2018-07-02 11:34:24.779219: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f500 of size 1024
2018-07-02 11:34:24.779232: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f900 of size 1024
2018-07-02 11:34:24.779245: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108fd00 of size 1024
2018-07-02 11:34:24.779257: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1090100 of size 126976
2018-07-02 11:34:24.779270: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10af100 of size 147456
2018-07-02 11:34:24.779283: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10d3100 of size 147456
2018-07-02 11:34:24.779296: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d10f7100 of size 294912
2018-07-02 11:34:24.779308: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d113f100 of size 147456
2018-07-02 11:34:24.779321: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1163100 of size 294912
2018-07-02 11:34:24.779333: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11ab100 of size 294912
2018-07-02 11:34:24.779347: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11f3100 of size 589824
2018-07-02 11:34:24.779359: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1283100 of size 589824
2018-07-02 11:34:24.779372: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1313100 of size 589824
2018-07-02 11:34:24.779384: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13a3100 of size 294912
2018-07-02 11:34:24.779398: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13eb100 of size 1141760
2018-07-02 11:34:24.779411: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1501d00 of size 9437184
2018-07-02 11:34:24.779424: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1e01d00 of size 9437184
2018-07-02 11:34:24.779437: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2701d00 of size 2048
2018-07-02 11:34:24.779449: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702500 of size 2048
2018-07-02 11:34:24.779462: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702d00 of size 2048
2018-07-02 11:34:24.779474: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703500 of size 2048
2018-07-02 11:34:24.779487: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703d00 of size 9437184
2018-07-02 11:34:24.788604: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3003d00 of size 9437184
2018-07-02 11:34:24.788652: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3903d00 of size 9437184
2018-07-02 11:34:24.788665: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4203d00 of size 9437184
2018-07-02 11:34:24.788675: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4b03d00 of size 9437184
2018-07-02 11:34:24.788686: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5403d00 of size 2048
2018-07-02 11:34:24.788695: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5404500 of size 4718592
2018-07-02 11:34:24.788706: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5884500 of size 2359296
2018-07-02 11:34:24.788716: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5ac4500 of size 2359296
2018-07-02 11:34:24.788725: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5d04500 of size 9437184
2018-07-02 11:34:24.788735: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6604500 of size 9437184
2018-07-02 11:34:24.788744: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6f04500 of size 9437184
2018-07-02 11:34:24.788753: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d7804500 of size 9437184
2018-07-02 11:34:24.788763: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8104500 of size 9437184
2018-07-02 11:34:24.788772: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8a04500 of size 9437184
2018-07-02 11:34:24.788782: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9304500 of size 9437184
2018-07-02 11:34:24.788791: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9c04500 of size 9437184
2018-07-02 11:34:24.788800: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da504500 of size 2359296
2018-07-02 11:34:24.788809: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da744500 of size 9437184
2018-07-02 11:34:24.788819: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1db044500 of size 15848704
2018-07-02 11:34:24.788829: I tensorflow/core/common_runtime/bfc_allocator.cc:671] Summary of in-use Chunks by size:
2018-07-02 11:34:24.788847: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 8 Chunks of size 256 totalling 2.0KiB
2018-07-02 11:34:24.788860: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 5 Chunks of size 512 totalling 2.5KiB
2018-07-02 11:34:24.788871: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 768 totalling 768B
2018-07-02 11:34:24.788882: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 6 Chunks of size 1024 totalling 6.0KiB
2018-07-02 11:34:24.788893: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1280 totalling 1.2KiB
2018-07-02 11:34:24.788906: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 23 Chunks of size 2048 totalling 46.0KiB
2018-07-02 11:34:24.788917: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 2816 totalling 2.8KiB
2018-07-02 11:34:24.788929: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 6912 totalling 20.2KiB
2018-07-02 11:34:24.788941: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 147456 totalling 432.0KiB
2018-07-02 11:34:24.788957: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 294912 totalling 864.0KiB
2018-07-02 11:34:24.788969: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 589824 totalling 1.12MiB
2018-07-02 11:34:24.788980: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1141760 totalling 1.09MiB
2018-07-02 11:34:24.788991: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 2359296 totalling 6.75MiB
2018-07-02 11:34:24.789002: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 4718592 totalling 9.00MiB
2018-07-02 11:34:24.789013: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9426944 totalling 8.99MiB
2018-07-02 11:34:24.789024: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 16 Chunks of size 9437184 totalling 144.00MiB
2018-07-02 11:34:24.789035: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9439232 totalling 9.00MiB
2018-07-02 11:34:24.789047: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 15848704 totalling 15.11MiB
2018-07-02 11:34:24.789059: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Sum Total of in-use chunks: 196.42MiB
2018-07-02 11:34:24.789073: I tensorflow/core/common_runtime/bfc_allocator.cc:680] Stats:
Limit: 229965824
InUse: 205956864
MaxInUse: 205956864
NumAllocs: 156
MaxAllocSize: 15848704

2018-07-02 11:34:24.789098: W tensorflow/core/common_runtime/bfc_allocator.cc:279] ***x*********************************************************************************************xxx
2018-07-02 11:34:24.789142: W tensorflow/core/framework/op_kernel.cc:1202] OP_REQUIRES failed at assign_op.h:111 : Resource exhausted: OOM when allocating tensor with shape[3,3,512,512] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
2018-07-02 11:34:24.790467: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.791427: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.791458: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 4.50MiB. Current allocation summary follows.
2018-07-02 11:34:24.791491: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (256): Total Chunks: 9, Chunks in use: 8. 2.2KiB allocated for chunks. 2.0KiB in use in bin. 1.5KiB client-requested in use in bin.
2018-07-02 11:34:24.791516: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (512): Total Chunks: 7, Chunks in use: 6. 3.8KiB allocated for chunks. 3.2KiB in use in bin. 3.0KiB client-requested in use in bin.
2018-07-02 11:34:24.791531: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1024): Total Chunks: 7, Chunks in use: 7. 7.2KiB allocated for chunks. 7.2KiB in use in bin. 7.0KiB client-requested in use in bin.
2018-07-02 11:34:24.791547: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2048): Total Chunks: 24, Chunks in use: 24. 48.8KiB allocated for chunks. 48.8KiB in use in bin. 48.0KiB client-requested in use in bin.
2018-07-02 11:34:24.791561: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4096): Total Chunks: 3, Chunks in use: 3. 20.2KiB allocated for chunks. 20.2KiB in use in bin. 20.2KiB client-requested in use in bin.
2018-07-02 11:34:24.791574: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.791587: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.791599: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.791619: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (65536): Total Chunks: 1, Chunks in use: 0. 124.0KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.791636: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (131072): Total Chunks: 3, Chunks in use: 3. 432.0KiB allocated for chunks. 432.0KiB in use in bin. 432.0KiB client-requested in use in bin.
2018-07-02 11:34:24.791651: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (262144): Total Chunks: 4, Chunks in use: 3. 1.12MiB allocated for chunks. 864.0KiB in use in bin. 864.0KiB client-requested in use in bin.
2018-07-02 11:34:24.791665: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (524288): Total Chunks: 3, Chunks in use: 2. 1.69MiB allocated for chunks. 1.12MiB in use in bin. 1.12MiB client-requested in use in bin.
2018-07-02 11:34:24.819634: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1048576): Total Chunks: 1, Chunks in use: 1. 1.09MiB allocated for chunks. 1.09MiB in use in bin. 576.0KiB client-requested in use in bin.
2018-07-02 11:34:24.819687: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2097152): Total Chunks: 3, Chunks in use: 2. 6.75MiB allocated for chunks. 4.50MiB in use in bin. 4.50MiB client-requested in use in bin.
2018-07-02 11:34:24.819716: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4194304): Total Chunks: 2, Chunks in use: 2. 9.00MiB allocated for chunks. 9.00MiB in use in bin. 9.00MiB client-requested in use in bin.
2018-07-02 11:34:24.819743: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8388608): Total Chunks: 19, Chunks in use: 19. 177.11MiB allocated for chunks. 177.11MiB in use in bin. 166.50MiB client-requested in use in bin.
2018-07-02 11:34:24.819759: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16777216): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.819771: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (33554432): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.819784: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.819797: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.819811: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.819831: I tensorflow/core/common_runtime/bfc_allocator.cc:646] Bin for 4.50MiB was 4.00MiB, Chunk State:
2018-07-02 11:34:24.819856: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00000 of size 1280
2018-07-02 11:34:24.819873: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00500 of size 256
2018-07-02 11:34:24.819887: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00600 of size 256
2018-07-02 11:34:24.819895: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00700 of size 256
2018-07-02 11:34:24.819903: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00800 of size 256
2018-07-02 11:34:24.819911: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00900 of size 256
2018-07-02 11:34:24.819919: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00a00 of size 256
2018-07-02 11:34:24.819926: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1cfa00b00 of size 256
2018-07-02 11:34:24.820082: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00c00 of size 256
2018-07-02 11:34:24.820102: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00d00 of size 2048
2018-07-02 11:34:24.820114: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01500 of size 2048
2018-07-02 11:34:24.820127: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01d00 of size 2048
2018-07-02 11:34:24.820142: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02500 of size 2048
2018-07-02 11:34:24.820159: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02d00 of size 9426944
2018-07-02 11:34:24.820176: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0300500 of size 9439232
2018-07-02 11:34:24.820192: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c00d00 of size 2048
2018-07-02 11:34:24.820208: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01500 of size 2048
2018-07-02 11:34:24.820226: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01d00 of size 4718592
2018-07-02 11:34:24.820243: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1081d00 of size 2048
2018-07-02 11:34:24.820259: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082500 of size 2048
2018-07-02 11:34:24.820275: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082d00 of size 256
2018-07-02 11:34:24.820290: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1082e00 of size 512
2018-07-02 11:34:24.820307: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083000 of size 512
2018-07-02 11:34:24.820324: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083200 of size 768
2018-07-02 11:34:24.820341: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083500 of size 2048
2018-07-02 11:34:24.820357: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083d00 of size 2048
2018-07-02 11:34:24.820375: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1084500 of size 6912
2018-07-02 11:34:24.820390: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086000 of size 512
2018-07-02 11:34:24.820404: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086200 of size 512
2018-07-02 11:34:24.820416: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086400 of size 512
2018-07-02 11:34:24.820429: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086600 of size 1024
2018-07-02 11:34:24.820441: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086a00 of size 1024
2018-07-02 11:34:24.820454: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086e00 of size 512
2018-07-02 11:34:24.820466: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087000 of size 2816
2018-07-02 11:34:24.820479: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087b00 of size 6912
2018-07-02 11:34:24.820491: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1089600 of size 6912
2018-07-02 11:34:24.820504: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b100 of size 2048
2018-07-02 11:34:24.820516: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b900 of size 2048
2018-07-02 11:34:24.820529: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c100 of size 2048
2018-07-02 11:34:24.820542: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c900 of size 2048
2018-07-02 11:34:24.820555: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d100 of size 2048
2018-07-02 11:34:24.820568: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d900 of size 2048
2018-07-02 11:34:24.820581: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e100 of size 2048
2018-07-02 11:34:24.820594: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e900 of size 1024
2018-07-02 11:34:24.820607: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108ed00 of size 2048
2018-07-02 11:34:24.820621: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f500 of size 1024
2018-07-02 11:34:24.820634: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f900 of size 1024
2018-07-02 11:34:24.820646: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108fd00 of size 1024
2018-07-02 11:34:24.829423: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1090100 of size 126976
2018-07-02 11:34:24.829462: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10af100 of size 147456
2018-07-02 11:34:24.829475: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10d3100 of size 147456
2018-07-02 11:34:24.829485: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d10f7100 of size 294912
2018-07-02 11:34:24.829495: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d113f100 of size 147456
2018-07-02 11:34:24.829505: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1163100 of size 294912
2018-07-02 11:34:24.829515: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11ab100 of size 294912
2018-07-02 11:34:24.829525: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11f3100 of size 589824
2018-07-02 11:34:24.829538: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1283100 of size 589824
2018-07-02 11:34:24.829549: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1313100 of size 589824
2018-07-02 11:34:24.829558: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13a3100 of size 294912
2018-07-02 11:34:24.829568: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13eb100 of size 1141760
2018-07-02 11:34:24.829578: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1501d00 of size 9437184
2018-07-02 11:34:24.829591: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1e01d00 of size 9437184
2018-07-02 11:34:24.829606: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2701d00 of size 2048
2018-07-02 11:34:24.829616: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702500 of size 2048
2018-07-02 11:34:24.829626: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702d00 of size 2048
2018-07-02 11:34:24.829635: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703500 of size 2048
2018-07-02 11:34:24.829645: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703d00 of size 9437184
2018-07-02 11:34:24.829654: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3003d00 of size 9437184
2018-07-02 11:34:24.829664: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3903d00 of size 9437184
2018-07-02 11:34:24.829673: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4203d00 of size 9437184
2018-07-02 11:34:24.829682: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4b03d00 of size 9437184
2018-07-02 11:34:24.829692: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5403d00 of size 2048
2018-07-02 11:34:24.829702: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5404500 of size 4718592
2018-07-02 11:34:24.829711: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d5884500 of size 2359296
2018-07-02 11:34:24.829721: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5ac4500 of size 2359296
2018-07-02 11:34:24.829731: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5d04500 of size 9437184
2018-07-02 11:34:24.829740: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6604500 of size 9437184
2018-07-02 11:34:24.829749: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6f04500 of size 9437184
2018-07-02 11:34:24.829759: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d7804500 of size 9437184
2018-07-02 11:34:24.829768: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8104500 of size 9437184
2018-07-02 11:34:24.829777: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8a04500 of size 9437184
2018-07-02 11:34:24.829787: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9304500 of size 9437184
2018-07-02 11:34:24.829796: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9c04500 of size 9437184
2018-07-02 11:34:24.829806: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da504500 of size 2359296
2018-07-02 11:34:24.829815: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da744500 of size 9437184
2018-07-02 11:34:24.829825: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1db044500 of size 15848704
2018-07-02 11:34:24.829834: I tensorflow/core/common_runtime/bfc_allocator.cc:671] Summary of in-use Chunks by size:
2018-07-02 11:34:24.829853: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 8 Chunks of size 256 totalling 2.0KiB
2018-07-02 11:34:24.829865: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 5 Chunks of size 512 totalling 2.5KiB
2018-07-02 11:34:24.829876: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 768 totalling 768B
2018-07-02 11:34:24.829887: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 6 Chunks of size 1024 totalling 6.0KiB
2018-07-02 11:34:24.829898: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1280 totalling 1.2KiB
2018-07-02 11:34:24.829912: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 23 Chunks of size 2048 totalling 46.0KiB
2018-07-02 11:34:24.829923: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 2816 totalling 2.8KiB
2018-07-02 11:34:24.829935: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 6912 totalling 20.2KiB
2018-07-02 11:34:24.829946: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 147456 totalling 432.0KiB
2018-07-02 11:34:24.829959: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 294912 totalling 864.0KiB
2018-07-02 11:34:24.829969: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 589824 totalling 1.12MiB
2018-07-02 11:34:24.829978: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1141760 totalling 1.09MiB
2018-07-02 11:34:24.829988: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 2359296 totalling 4.50MiB
2018-07-02 11:34:24.830006: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 4718592 totalling 9.00MiB
2018-07-02 11:34:24.830018: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9426944 totalling 8.99MiB
2018-07-02 11:34:24.830030: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 16 Chunks of size 9437184 totalling 144.00MiB
2018-07-02 11:34:24.830041: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9439232 totalling 9.00MiB
2018-07-02 11:34:24.830053: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 15848704 totalling 15.11MiB
2018-07-02 11:34:24.830065: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Sum Total of in-use chunks: 194.17MiB
2018-07-02 11:34:24.830084: I tensorflow/core/common_runtime/bfc_allocator.cc:680] Stats:
Limit: 229965824
InUse: 203597568
MaxInUse: 205956864
NumAllocs: 156
MaxAllocSize: 15848704

2018-07-02 11:34:24.830116: W tensorflow/core/common_runtime/bfc_allocator.cc:279] ***x********************************************_************************************************xxx
2018-07-02 11:34:24.831201: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.832169: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.832194: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 9.00MiB. Current allocation summary follows.
2018-07-02 11:34:24.832214: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (256): Total Chunks: 9, Chunks in use: 8. 2.2KiB allocated for chunks. 2.0KiB in use in bin. 1.5KiB client-requested in use in bin.
2018-07-02 11:34:24.849839: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (512): Total Chunks: 7, Chunks in use: 6. 3.8KiB allocated for chunks. 3.2KiB in use in bin. 3.0KiB client-requested in use in bin.
2018-07-02 11:34:24.849888: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1024): Total Chunks: 7, Chunks in use: 7. 7.2KiB allocated for chunks. 7.2KiB in use in bin. 7.0KiB client-requested in use in bin.
2018-07-02 11:34:24.849913: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2048): Total Chunks: 24, Chunks in use: 24. 48.8KiB allocated for chunks. 48.8KiB in use in bin. 48.0KiB client-requested in use in bin.
2018-07-02 11:34:24.849929: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4096): Total Chunks: 3, Chunks in use: 3. 20.2KiB allocated for chunks. 20.2KiB in use in bin. 20.2KiB client-requested in use in bin.
2018-07-02 11:34:24.849942: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.849955: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.849967: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.849981: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (65536): Total Chunks: 1, Chunks in use: 0. 124.0KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.849996: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (131072): Total Chunks: 3, Chunks in use: 3. 432.0KiB allocated for chunks. 432.0KiB in use in bin. 432.0KiB client-requested in use in bin.
2018-07-02 11:34:24.850010: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (262144): Total Chunks: 4, Chunks in use: 3. 1.12MiB allocated for chunks. 864.0KiB in use in bin. 864.0KiB client-requested in use in bin.
2018-07-02 11:34:24.850024: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (524288): Total Chunks: 3, Chunks in use: 2. 1.69MiB allocated for chunks. 1.12MiB in use in bin. 1.12MiB client-requested in use in bin.
2018-07-02 11:34:24.850038: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1048576): Total Chunks: 1, Chunks in use: 1. 1.09MiB allocated for chunks. 1.09MiB in use in bin. 576.0KiB client-requested in use in bin.
2018-07-02 11:34:24.850051: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2097152): Total Chunks: 3, Chunks in use: 2. 6.75MiB allocated for chunks. 4.50MiB in use in bin. 4.50MiB client-requested in use in bin.
2018-07-02 11:34:24.850065: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4194304): Total Chunks: 2, Chunks in use: 2. 9.00MiB allocated for chunks. 9.00MiB in use in bin. 9.00MiB client-requested in use in bin.
2018-07-02 11:34:24.850080: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8388608): Total Chunks: 19, Chunks in use: 19. 177.11MiB allocated for chunks. 177.11MiB in use in bin. 166.50MiB client-requested in use in bin.
2018-07-02 11:34:24.850093: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16777216): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.850105: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (33554432): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.850118: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.850130: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.850142: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.850155: I tensorflow/core/common_runtime/bfc_allocator.cc:646] Bin for 9.00MiB was 8.00MiB, Chunk State:
2018-07-02 11:34:24.850175: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00000 of size 1280
2018-07-02 11:34:24.850193: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00500 of size 256
2018-07-02 11:34:24.850208: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00600 of size 256
2018-07-02 11:34:24.850221: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00700 of size 256
2018-07-02 11:34:24.850233: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00800 of size 256
2018-07-02 11:34:24.850246: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00900 of size 256
2018-07-02 11:34:24.850258: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00a00 of size 256
2018-07-02 11:34:24.850271: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1cfa00b00 of size 256
2018-07-02 11:34:24.850283: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00c00 of size 256
2018-07-02 11:34:24.850296: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00d00 of size 2048
2018-07-02 11:34:24.850308: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01500 of size 2048
2018-07-02 11:34:24.850323: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01d00 of size 2048
2018-07-02 11:34:24.850340: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02500 of size 2048
2018-07-02 11:34:24.850353: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02d00 of size 9426944
2018-07-02 11:34:24.850365: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0300500 of size 9439232
2018-07-02 11:34:24.850378: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c00d00 of size 2048
2018-07-02 11:34:24.850390: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01500 of size 2048
2018-07-02 11:34:24.850423: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01d00 of size 4718592
2018-07-02 11:34:24.850436: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1081d00 of size 2048
2018-07-02 11:34:24.850449: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082500 of size 2048
2018-07-02 11:34:24.850461: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082d00 of size 256
2018-07-02 11:34:24.850473: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1082e00 of size 512
2018-07-02 11:34:24.850486: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083000 of size 512
2018-07-02 11:34:24.850499: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083200 of size 768
2018-07-02 11:34:24.850511: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083500 of size 2048
2018-07-02 11:34:24.850523: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083d00 of size 2048
2018-07-02 11:34:24.850536: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1084500 of size 6912
2018-07-02 11:34:24.858730: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086000 of size 512
2018-07-02 11:34:24.858776: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086200 of size 512
2018-07-02 11:34:24.858793: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086400 of size 512
2018-07-02 11:34:24.858810: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086600 of size 1024
2018-07-02 11:34:24.858825: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086a00 of size 1024
2018-07-02 11:34:24.858838: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086e00 of size 512
2018-07-02 11:34:24.858855: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087000 of size 2816
2018-07-02 11:34:24.858877: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087b00 of size 6912
2018-07-02 11:34:24.858893: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1089600 of size 6912
2018-07-02 11:34:24.858909: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b100 of size 2048
2018-07-02 11:34:24.858924: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b900 of size 2048
2018-07-02 11:34:24.858938: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c100 of size 2048
2018-07-02 11:34:24.858953: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c900 of size 2048
2018-07-02 11:34:24.858968: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d100 of size 2048
2018-07-02 11:34:24.858982: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d900 of size 2048
2018-07-02 11:34:24.858997: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e100 of size 2048
2018-07-02 11:34:24.859012: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e900 of size 1024
2018-07-02 11:34:24.859027: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108ed00 of size 2048
2018-07-02 11:34:24.859043: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f500 of size 1024
2018-07-02 11:34:24.859058: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f900 of size 1024
2018-07-02 11:34:24.859074: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108fd00 of size 1024
2018-07-02 11:34:24.859088: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1090100 of size 126976
2018-07-02 11:34:24.859104: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10af100 of size 147456
2018-07-02 11:34:24.859118: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10d3100 of size 147456
2018-07-02 11:34:24.859132: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d10f7100 of size 294912
2018-07-02 11:34:24.859145: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d113f100 of size 147456
2018-07-02 11:34:24.859159: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1163100 of size 294912
2018-07-02 11:34:24.859172: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11ab100 of size 294912
2018-07-02 11:34:24.859186: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11f3100 of size 589824
2018-07-02 11:34:24.859199: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1283100 of size 589824
2018-07-02 11:34:24.859211: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1313100 of size 589824
2018-07-02 11:34:24.859226: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13a3100 of size 294912
2018-07-02 11:34:24.859242: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13eb100 of size 1141760
2018-07-02 11:34:24.859257: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1501d00 of size 9437184
2018-07-02 11:34:24.859272: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1e01d00 of size 9437184
2018-07-02 11:34:24.859286: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2701d00 of size 2048
2018-07-02 11:34:24.859301: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702500 of size 2048
2018-07-02 11:34:24.859315: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702d00 of size 2048
2018-07-02 11:34:24.859330: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703500 of size 2048
2018-07-02 11:34:24.859344: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703d00 of size 9437184
2018-07-02 11:34:24.859358: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3003d00 of size 9437184
2018-07-02 11:34:24.859372: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3903d00 of size 9437184
2018-07-02 11:34:24.859386: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4203d00 of size 9437184
2018-07-02 11:34:24.859405: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4b03d00 of size 9437184
2018-07-02 11:34:24.859420: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5403d00 of size 2048
2018-07-02 11:34:24.859434: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5404500 of size 4718592
2018-07-02 11:34:24.859448: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d5884500 of size 2359296
2018-07-02 11:34:24.859463: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5ac4500 of size 2359296
2018-07-02 11:34:24.859478: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5d04500 of size 9437184
2018-07-02 11:34:24.859492: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6604500 of size 9437184
2018-07-02 11:34:24.859506: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6f04500 of size 9437184
2018-07-02 11:34:24.859521: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d7804500 of size 9437184
2018-07-02 11:34:24.859537: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8104500 of size 9437184
2018-07-02 11:34:24.859551: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8a04500 of size 9437184
2018-07-02 11:34:24.859567: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9304500 of size 9437184
2018-07-02 11:34:24.859582: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9c04500 of size 9437184
2018-07-02 11:34:24.859598: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da504500 of size 2359296
2018-07-02 11:34:24.859612: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da744500 of size 9437184
2018-07-02 11:34:24.859628: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1db044500 of size 15848704
2018-07-02 11:34:24.859642: I tensorflow/core/common_runtime/bfc_allocator.cc:671] Summary of in-use Chunks by size:
2018-07-02 11:34:24.859667: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 8 Chunks of size 256 totalling 2.0KiB
2018-07-02 11:34:24.859685: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 5 Chunks of size 512 totalling 2.5KiB
2018-07-02 11:34:24.859703: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 768 totalling 768B
2018-07-02 11:34:24.859720: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 6 Chunks of size 1024 totalling 6.0KiB
2018-07-02 11:34:24.859739: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1280 totalling 1.2KiB
2018-07-02 11:34:24.859759: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 23 Chunks of size 2048 totalling 46.0KiB
2018-07-02 11:34:24.859776: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 2816 totalling 2.8KiB
2018-07-02 11:34:24.859795: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 6912 totalling 20.2KiB
2018-07-02 11:34:24.880622: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 147456 totalling 432.0KiB
2018-07-02 11:34:24.880664: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 294912 totalling 864.0KiB
2018-07-02 11:34:24.880681: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 589824 totalling 1.12MiB
2018-07-02 11:34:24.880693: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1141760 totalling 1.09MiB
2018-07-02 11:34:24.880705: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 2359296 totalling 4.50MiB
2018-07-02 11:34:24.880716: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 4718592 totalling 9.00MiB
2018-07-02 11:34:24.880727: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9426944 totalling 8.99MiB
2018-07-02 11:34:24.880739: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 16 Chunks of size 9437184 totalling 144.00MiB
2018-07-02 11:34:24.880751: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9439232 totalling 9.00MiB
2018-07-02 11:34:24.880763: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 15848704 totalling 15.11MiB
2018-07-02 11:34:24.880775: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Sum Total of in-use chunks: 194.17MiB
2018-07-02 11:34:24.880790: I tensorflow/core/common_runtime/bfc_allocator.cc:680] Stats:
Limit: 229965824
InUse: 203597568
MaxInUse: 205956864
NumAllocs: 156
MaxAllocSize: 15848704

2018-07-02 11:34:24.880819: W tensorflow/core/common_runtime/bfc_allocator.cc:279] ***x********************************************_************************************************xxx
2018-07-02 11:34:24.882008: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.883272: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.883330: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 4.50MiB. Current allocation summary follows.
2018-07-02 11:34:24.883357: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (256): Total Chunks: 9, Chunks in use: 8. 2.2KiB allocated for chunks. 2.0KiB in use in bin. 1.5KiB client-requested in use in bin.
2018-07-02 11:34:24.883371: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (512): Total Chunks: 7, Chunks in use: 6. 3.8KiB allocated for chunks. 3.2KiB in use in bin. 3.0KiB client-requested in use in bin.
2018-07-02 11:34:24.883384: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1024): Total Chunks: 7, Chunks in use: 7. 7.2KiB allocated for chunks. 7.2KiB in use in bin. 7.0KiB client-requested in use in bin.
2018-07-02 11:34:24.883397: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2048): Total Chunks: 24, Chunks in use: 24. 48.8KiB allocated for chunks. 48.8KiB in use in bin. 48.0KiB client-requested in use in bin.
2018-07-02 11:34:24.883411: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4096): Total Chunks: 3, Chunks in use: 3. 20.2KiB allocated for chunks. 20.2KiB in use in bin. 20.2KiB client-requested in use in bin.
2018-07-02 11:34:24.883422: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.883434: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.883445: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.883457: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (65536): Total Chunks: 1, Chunks in use: 0. 124.0KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.883470: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (131072): Total Chunks: 3, Chunks in use: 3. 432.0KiB allocated for chunks. 432.0KiB in use in bin. 432.0KiB client-requested in use in bin.
2018-07-02 11:34:24.883483: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (262144): Total Chunks: 4, Chunks in use: 3. 1.12MiB allocated for chunks. 864.0KiB in use in bin. 864.0KiB client-requested in use in bin.
2018-07-02 11:34:24.883495: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (524288): Total Chunks: 3, Chunks in use: 2. 1.69MiB allocated for chunks. 1.12MiB in use in bin. 1.12MiB client-requested in use in bin.
2018-07-02 11:34:24.883507: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1048576): Total Chunks: 1, Chunks in use: 1. 1.09MiB allocated for chunks. 1.09MiB in use in bin. 576.0KiB client-requested in use in bin.
2018-07-02 11:34:24.883519: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2097152): Total Chunks: 3, Chunks in use: 2. 6.75MiB allocated for chunks. 4.50MiB in use in bin. 4.50MiB client-requested in use in bin.
2018-07-02 11:34:24.883531: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4194304): Total Chunks: 2, Chunks in use: 2. 9.00MiB allocated for chunks. 9.00MiB in use in bin. 9.00MiB client-requested in use in bin.
2018-07-02 11:34:24.883545: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8388608): Total Chunks: 19, Chunks in use: 19. 177.11MiB allocated for chunks. 177.11MiB in use in bin. 166.50MiB client-requested in use in bin.
2018-07-02 11:34:24.883556: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16777216): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.883567: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (33554432): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.883578: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.883591: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.883609: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.883622: I tensorflow/core/common_runtime/bfc_allocator.cc:646] Bin for 4.50MiB was 4.00MiB, Chunk State:
2018-07-02 11:34:24.883634: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00000 of size 1280
2018-07-02 11:34:24.883643: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00500 of size 256
2018-07-02 11:34:24.883651: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00600 of size 256
2018-07-02 11:34:24.883659: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00700 of size 256
2018-07-02 11:34:24.883667: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00800 of size 256
2018-07-02 11:34:24.897197: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00900 of size 256
2018-07-02 11:34:24.897237: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00a00 of size 256
2018-07-02 11:34:24.897254: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1cfa00b00 of size 256
2018-07-02 11:34:24.897268: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00c00 of size 256
2018-07-02 11:34:24.897283: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00d00 of size 2048
2018-07-02 11:34:24.897297: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01500 of size 2048
2018-07-02 11:34:24.897311: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01d00 of size 2048
2018-07-02 11:34:24.897327: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02500 of size 2048
2018-07-02 11:34:24.897357: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02d00 of size 9426944
2018-07-02 11:34:24.897366: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0300500 of size 9439232
2018-07-02 11:34:24.897373: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c00d00 of size 2048
2018-07-02 11:34:24.897381: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01500 of size 2048
2018-07-02 11:34:24.897389: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01d00 of size 4718592
2018-07-02 11:34:24.897397: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1081d00 of size 2048
2018-07-02 11:34:24.897404: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082500 of size 2048
2018-07-02 11:34:24.897412: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082d00 of size 256
2018-07-02 11:34:24.897419: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1082e00 of size 512
2018-07-02 11:34:24.897427: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083000 of size 512
2018-07-02 11:34:24.897435: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083200 of size 768
2018-07-02 11:34:24.897443: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083500 of size 2048
2018-07-02 11:34:24.897450: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083d00 of size 2048
2018-07-02 11:34:24.897458: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1084500 of size 6912
2018-07-02 11:34:24.897466: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086000 of size 512
2018-07-02 11:34:24.897473: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086200 of size 512
2018-07-02 11:34:24.897481: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086400 of size 512
2018-07-02 11:34:24.897489: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086600 of size 1024
2018-07-02 11:34:24.897497: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086a00 of size 1024
2018-07-02 11:34:24.897504: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086e00 of size 512
2018-07-02 11:34:24.897512: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087000 of size 2816
2018-07-02 11:34:24.897520: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087b00 of size 6912
2018-07-02 11:34:24.897527: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1089600 of size 6912
2018-07-02 11:34:24.897540: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b100 of size 2048
2018-07-02 11:34:24.897548: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b900 of size 2048
2018-07-02 11:34:24.897556: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c100 of size 2048
2018-07-02 11:34:24.897563: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c900 of size 2048
2018-07-02 11:34:24.897571: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d100 of size 2048
2018-07-02 11:34:24.897578: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d900 of size 2048
2018-07-02 11:34:24.897586: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e100 of size 2048
2018-07-02 11:34:24.897593: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e900 of size 1024
2018-07-02 11:34:24.897601: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108ed00 of size 2048
2018-07-02 11:34:24.897609: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f500 of size 1024
2018-07-02 11:34:24.897616: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f900 of size 1024
2018-07-02 11:34:24.897624: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108fd00 of size 1024
2018-07-02 11:34:24.897631: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1090100 of size 126976
2018-07-02 11:34:24.897639: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10af100 of size 147456
2018-07-02 11:34:24.897647: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10d3100 of size 147456
2018-07-02 11:34:24.897654: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d10f7100 of size 294912
2018-07-02 11:34:24.897662: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d113f100 of size 147456
2018-07-02 11:34:24.897669: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1163100 of size 294912
2018-07-02 11:34:24.897677: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11ab100 of size 294912
2018-07-02 11:34:24.897685: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11f3100 of size 589824
2018-07-02 11:34:24.897692: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1283100 of size 589824
2018-07-02 11:34:24.897700: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1313100 of size 589824
2018-07-02 11:34:24.897707: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13a3100 of size 294912
2018-07-02 11:34:24.897715: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13eb100 of size 1141760
2018-07-02 11:34:24.897723: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1501d00 of size 9437184
2018-07-02 11:34:24.897731: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1e01d00 of size 9437184
2018-07-02 11:34:24.897738: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2701d00 of size 2048
2018-07-02 11:34:24.897746: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702500 of size 2048
2018-07-02 11:34:24.897753: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702d00 of size 2048
2018-07-02 11:34:24.897761: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703500 of size 2048
2018-07-02 11:34:24.897768: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703d00 of size 9437184
2018-07-02 11:34:24.897775: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3003d00 of size 9437184
2018-07-02 11:34:24.897783: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3903d00 of size 9437184
2018-07-02 11:34:24.897790: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4203d00 of size 9437184
2018-07-02 11:34:24.897798: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4b03d00 of size 9437184
2018-07-02 11:34:24.897805: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5403d00 of size 2048
2018-07-02 11:34:24.897813: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5404500 of size 4718592
2018-07-02 11:34:24.910885: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d5884500 of size 2359296
2018-07-02 11:34:24.910914: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5ac4500 of size 2359296
2018-07-02 11:34:24.910924: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5d04500 of size 9437184
2018-07-02 11:34:24.910932: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6604500 of size 9437184
2018-07-02 11:34:24.910946: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6f04500 of size 9437184
2018-07-02 11:34:24.910957: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d7804500 of size 9437184
2018-07-02 11:34:24.910966: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8104500 of size 9437184
2018-07-02 11:34:24.910976: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8a04500 of size 9437184
2018-07-02 11:34:24.910985: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9304500 of size 9437184
2018-07-02 11:34:24.910997: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9c04500 of size 9437184
2018-07-02 11:34:24.911009: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da504500 of size 2359296
2018-07-02 11:34:24.911020: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da744500 of size 9437184
2018-07-02 11:34:24.911031: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1db044500 of size 15848704
2018-07-02 11:34:24.911042: I tensorflow/core/common_runtime/bfc_allocator.cc:671] Summary of in-use Chunks by size:
2018-07-02 11:34:24.911061: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 8 Chunks of size 256 totalling 2.0KiB
2018-07-02 11:34:24.911076: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 5 Chunks of size 512 totalling 2.5KiB
2018-07-02 11:34:24.911092: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 768 totalling 768B
2018-07-02 11:34:24.911103: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 6 Chunks of size 1024 totalling 6.0KiB
2018-07-02 11:34:24.911112: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1280 totalling 1.2KiB
2018-07-02 11:34:24.911122: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 23 Chunks of size 2048 totalling 46.0KiB
2018-07-02 11:34:24.911131: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 2816 totalling 2.8KiB
2018-07-02 11:34:24.911140: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 6912 totalling 20.2KiB
2018-07-02 11:34:24.911152: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 147456 totalling 432.0KiB
2018-07-02 11:34:24.911166: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 294912 totalling 864.0KiB
2018-07-02 11:34:24.911177: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 589824 totalling 1.12MiB
2018-07-02 11:34:24.911185: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1141760 totalling 1.09MiB
2018-07-02 11:34:24.911194: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 2359296 totalling 4.50MiB
2018-07-02 11:34:24.911203: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 4718592 totalling 9.00MiB
2018-07-02 11:34:24.911211: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9426944 totalling 8.99MiB
2018-07-02 11:34:24.911220: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 16 Chunks of size 9437184 totalling 144.00MiB
2018-07-02 11:34:24.911229: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9439232 totalling 9.00MiB
2018-07-02 11:34:24.911238: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 15848704 totalling 15.11MiB
2018-07-02 11:34:24.911247: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Sum Total of in-use chunks: 194.17MiB
2018-07-02 11:34:24.911259: I tensorflow/core/common_runtime/bfc_allocator.cc:680] Stats:
Limit: 229965824
InUse: 203597568
MaxInUse: 205956864
NumAllocs: 156
MaxAllocSize: 15848704

2018-07-02 11:34:24.911284: W tensorflow/core/common_runtime/bfc_allocator.cc:279] ***x********************************************_************************************************xxx
2018-07-02 11:34:24.913143: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.914557: E tensorflow/stream_executor/cuda/cuda_driver.cc:936] failed to allocate 21.93M (22996480 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY
2018-07-02 11:34:24.914597: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 9.00MiB. Current allocation summary follows.
2018-07-02 11:34:24.914624: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (256): Total Chunks: 9, Chunks in use: 8. 2.2KiB allocated for chunks. 2.0KiB in use in bin. 1.5KiB client-requested in use in bin.
2018-07-02 11:34:24.914646: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (512): Total Chunks: 7, Chunks in use: 6. 3.8KiB allocated for chunks. 3.2KiB in use in bin. 3.0KiB client-requested in use in bin.
2018-07-02 11:34:24.914664: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1024): Total Chunks: 7, Chunks in use: 7. 7.2KiB allocated for chunks. 7.2KiB in use in bin. 7.0KiB client-requested in use in bin.
2018-07-02 11:34:24.914683: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2048): Total Chunks: 24, Chunks in use: 24. 48.8KiB allocated for chunks. 48.8KiB in use in bin. 48.0KiB client-requested in use in bin.
2018-07-02 11:34:24.914702: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4096): Total Chunks: 3, Chunks in use: 3. 20.2KiB allocated for chunks. 20.2KiB in use in bin. 20.2KiB client-requested in use in bin.
2018-07-02 11:34:24.914719: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.914735: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.914752: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.914769: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (65536): Total Chunks: 1, Chunks in use: 0. 124.0KiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.914788: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (131072): Total Chunks: 3, Chunks in use: 3. 432.0KiB allocated for chunks. 432.0KiB in use in bin. 432.0KiB client-requested in use in bin.
2018-07-02 11:34:24.914807: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (262144): Total Chunks: 4, Chunks in use: 3. 1.12MiB allocated for chunks. 864.0KiB in use in bin. 864.0KiB client-requested in use in bin.
2018-07-02 11:34:24.914825: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (524288): Total Chunks: 3, Chunks in use: 2. 1.69MiB allocated for chunks. 1.12MiB in use in bin. 1.12MiB client-requested in use in bin.
2018-07-02 11:34:24.914843: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (1048576): Total Chunks: 1, Chunks in use: 1. 1.09MiB allocated for chunks. 1.09MiB in use in bin. 576.0KiB client-requested in use in bin.
2018-07-02 11:34:24.914861: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (2097152): Total Chunks: 3, Chunks in use: 2. 6.75MiB allocated for chunks. 4.50MiB in use in bin. 4.50MiB client-requested in use in bin.
2018-07-02 11:34:24.914879: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (4194304): Total Chunks: 2, Chunks in use: 2. 9.00MiB allocated for chunks. 9.00MiB in use in bin. 9.00MiB client-requested in use in bin.
2018-07-02 11:34:24.919186: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (8388608): Total Chunks: 19, Chunks in use: 19. 177.11MiB allocated for chunks. 177.11MiB in use in bin. 166.50MiB client-requested in use in bin.
2018-07-02 11:34:24.919232: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (16777216): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.919253: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (33554432): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.919269: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.919286: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.919303: I tensorflow/core/common_runtime/bfc_allocator.cc:630] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2018-07-02 11:34:24.919341: I tensorflow/core/common_runtime/bfc_allocator.cc:646] Bin for 9.00MiB was 8.00MiB, Chunk State:
2018-07-02 11:34:24.919357: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00000 of size 1280
2018-07-02 11:34:24.919368: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00500 of size 256
2018-07-02 11:34:24.919377: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00600 of size 256
2018-07-02 11:34:24.919387: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00700 of size 256
2018-07-02 11:34:24.919396: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00800 of size 256
2018-07-02 11:34:24.919406: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00900 of size 256
2018-07-02 11:34:24.919425: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00a00 of size 256
2018-07-02 11:34:24.919445: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1cfa00b00 of size 256
2018-07-02 11:34:24.919455: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00c00 of size 256
2018-07-02 11:34:24.919464: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa00d00 of size 2048
2018-07-02 11:34:24.919471: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01500 of size 2048
2018-07-02 11:34:24.919479: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa01d00 of size 2048
2018-07-02 11:34:24.919487: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02500 of size 2048
2018-07-02 11:34:24.919495: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1cfa02d00 of size 9426944
2018-07-02 11:34:24.919503: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0300500 of size 9439232
2018-07-02 11:34:24.919514: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c00d00 of size 2048
2018-07-02 11:34:24.919524: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01500 of size 2048
2018-07-02 11:34:24.919536: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d0c01d00 of size 4718592
2018-07-02 11:34:24.919546: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1081d00 of size 2048
2018-07-02 11:34:24.919556: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082500 of size 2048
2018-07-02 11:34:24.919567: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1082d00 of size 256
2018-07-02 11:34:24.919575: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1082e00 of size 512
2018-07-02 11:34:24.919583: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083000 of size 512
2018-07-02 11:34:24.919592: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083200 of size 768
2018-07-02 11:34:24.919599: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083500 of size 2048
2018-07-02 11:34:24.919607: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1083d00 of size 2048
2018-07-02 11:34:24.919634: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1084500 of size 6912
2018-07-02 11:34:24.919647: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086000 of size 512
2018-07-02 11:34:24.919658: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086200 of size 512
2018-07-02 11:34:24.919670: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086400 of size 512
2018-07-02 11:34:24.919681: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086600 of size 1024
2018-07-02 11:34:24.919693: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086a00 of size 1024
2018-07-02 11:34:24.919704: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1086e00 of size 512
2018-07-02 11:34:24.919717: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087000 of size 2816
2018-07-02 11:34:24.919730: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1087b00 of size 6912
2018-07-02 11:34:24.919742: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1089600 of size 6912
2018-07-02 11:34:24.919755: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b100 of size 2048
2018-07-02 11:34:24.919767: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108b900 of size 2048
2018-07-02 11:34:24.919778: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c100 of size 2048
2018-07-02 11:34:24.919796: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108c900 of size 2048
2018-07-02 11:34:24.919814: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d100 of size 2048
2018-07-02 11:34:24.919827: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108d900 of size 2048
2018-07-02 11:34:24.919839: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e100 of size 2048
2018-07-02 11:34:24.919850: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108e900 of size 1024
2018-07-02 11:34:24.919870: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108ed00 of size 2048
2018-07-02 11:34:24.919888: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f500 of size 1024
2018-07-02 11:34:24.919898: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108f900 of size 1024
2018-07-02 11:34:24.919908: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d108fd00 of size 1024
2018-07-02 11:34:24.919917: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1090100 of size 126976
2018-07-02 11:34:24.919927: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10af100 of size 147456
2018-07-02 11:34:24.919936: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d10d3100 of size 147456
2018-07-02 11:34:24.919946: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d10f7100 of size 294912
2018-07-02 11:34:24.919955: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d113f100 of size 147456
2018-07-02 11:34:24.919968: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1163100 of size 294912
2018-07-02 11:34:24.919978: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11ab100 of size 294912
2018-07-02 11:34:24.941208: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d11f3100 of size 589824
2018-07-02 11:34:24.941247: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d1283100 of size 589824
2018-07-02 11:34:24.941261: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1313100 of size 589824
2018-07-02 11:34:24.941273: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13a3100 of size 294912
2018-07-02 11:34:24.941291: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d13eb100 of size 1141760
2018-07-02 11:34:24.941304: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1501d00 of size 9437184
2018-07-02 11:34:24.941315: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d1e01d00 of size 9437184
2018-07-02 11:34:24.941327: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2701d00 of size 2048
2018-07-02 11:34:24.941339: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702500 of size 2048
2018-07-02 11:34:24.941350: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2702d00 of size 2048
2018-07-02 11:34:24.941361: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703500 of size 2048
2018-07-02 11:34:24.941372: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d2703d00 of size 9437184
2018-07-02 11:34:24.941384: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3003d00 of size 9437184
2018-07-02 11:34:24.941396: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d3903d00 of size 9437184
2018-07-02 11:34:24.941407: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4203d00 of size 9437184
2018-07-02 11:34:24.941418: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d4b03d00 of size 9437184
2018-07-02 11:34:24.941430: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5403d00 of size 2048
2018-07-02 11:34:24.941441: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5404500 of size 4718592
2018-07-02 11:34:24.941453: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Free at 0x7fc1d5884500 of size 2359296
2018-07-02 11:34:24.941465: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5ac4500 of size 2359296
2018-07-02 11:34:24.941476: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d5d04500 of size 9437184
2018-07-02 11:34:24.941487: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6604500 of size 9437184
2018-07-02 11:34:24.941499: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d6f04500 of size 9437184
2018-07-02 11:34:24.941510: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d7804500 of size 9437184
2018-07-02 11:34:24.941521: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8104500 of size 9437184
2018-07-02 11:34:24.941534: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d8a04500 of size 9437184
2018-07-02 11:34:24.941546: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9304500 of size 9437184
2018-07-02 11:34:24.941558: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1d9c04500 of size 9437184
2018-07-02 11:34:24.941570: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da504500 of size 2359296
2018-07-02 11:34:24.941583: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1da744500 of size 9437184
2018-07-02 11:34:24.941596: I tensorflow/core/common_runtime/bfc_allocator.cc:665] Chunk at 0x7fc1db044500 of size 15848704
2018-07-02 11:34:24.941607: I tensorflow/core/common_runtime/bfc_allocator.cc:671] Summary of in-use Chunks by size:
2018-07-02 11:34:24.941627: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 8 Chunks of size 256 totalling 2.0KiB
2018-07-02 11:34:24.941643: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 5 Chunks of size 512 totalling 2.5KiB
2018-07-02 11:34:24.941657: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 768 totalling 768B
2018-07-02 11:34:24.941671: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 6 Chunks of size 1024 totalling 6.0KiB
2018-07-02 11:34:24.941685: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1280 totalling 1.2KiB
2018-07-02 11:34:24.941701: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 23 Chunks of size 2048 totalling 46.0KiB
2018-07-02 11:34:24.941715: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 2816 totalling 2.8KiB
2018-07-02 11:34:24.941730: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 6912 totalling 20.2KiB
2018-07-02 11:34:24.941744: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 147456 totalling 432.0KiB
2018-07-02 11:34:24.941760: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 3 Chunks of size 294912 totalling 864.0KiB
2018-07-02 11:34:24.941774: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 589824 totalling 1.12MiB
2018-07-02 11:34:24.941789: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 1141760 totalling 1.09MiB
2018-07-02 11:34:24.941803: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 2359296 totalling 4.50MiB
2018-07-02 11:34:24.941817: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 2 Chunks of size 4718592 totalling 9.00MiB
2018-07-02 11:34:24.941831: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9426944 totalling 8.99MiB
2018-07-02 11:34:24.941846: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 16 Chunks of size 9437184 totalling 144.00MiB
2018-07-02 11:34:24.941860: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 9439232 totalling 9.00MiB
2018-07-02 11:34:24.941875: I tensorflow/core/common_runtime/bfc_allocator.cc:674] 1 Chunks of size 15848704 totalling 15.11MiB
2018-07-02 11:34:24.941890: I tensorflow/core/common_runtime/bfc_allocator.cc:678] Sum Total of in-use chunks: 194.17MiB
2018-07-02 11:34:24.941907: I tensorflow/core/common_runtime/bfc_allocator.cc:680] Stats:
Limit: 229965824
InUse: 203597568
MaxInUse: 205956864
NumAllocs: 156
MaxAllocSize: 15848704

2018-07-02 11:34:24.941938: W tensorflow/core/common_runtime/bfc_allocator.cc:279] ***x********************************************_************************************************xxx
2018-07-02 11:34:24,942 [Thread-7 ] [hbp_nrp_cle.] [ERROR] Error while loading new transfer function
2018-07-02 11:34:24,942 [Thread-7 ] [hbp_nrp_cle.] [ERROR] Dst tensor is not initialized.
[[Node: save/RestoreV2_30/_63 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_322_save/RestoreV2_30", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
[[Node: save/RestoreV2_43/_44 = _Send[T=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_304_save/RestoreV2_43", _device="/job:localhost/replica:0/task:0/device:CPU:0"](save/RestoreV2_43)]]
Traceback (most recent call last):
File "/home/alban/Documents/NRP/CLE/hbp_nrp_cle/hbp_nrp_cle/tf_framework/__init__.py", line 404, in set_transfer_function
exec new_code
File "<string>", line 12, in <module>
File "build/bdist.linux-x86_64/egg/attention/saliency.py", line 66, in __init__
self.net.restore(self.sess, model_file)
File "/home/alban/.opt/platform_venv/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1755, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/alban/.opt/platform_venv/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 905, in run
run_metadata_ptr)
File "/home/alban/.opt/platform_venv/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1137, in _run
feed_dict_tensor, options, run_metadata)
File "/home/alban/.opt/platform_venv/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1355, in _do_run
options, run_metadata)
File "/home/alban/.opt/platform_venv/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1374, in _do_call
raise type(e)(node_def, op, message)
InternalError: Dst tensor is not initialized.
[[Node: save/RestoreV2_30/_63 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_322_save/RestoreV2_30", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
[[Node: save/RestoreV2_43/_44 = _Send[T=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_304_save/RestoreV2_43", _device="/job:localhost/replica:0/task:0/device:CPU:0"](save/RestoreV2_43)]]
2018-07-02 11:34:24,948 [Thread-7 ] [hbp_nrp_cles] [ERROR] image_to_saliency: 'Dst tensor is not initialized.\n\t [[Node: save/RestoreV2_30/_63 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_322_save/RestoreV2_30", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]\n\t [[Node: save/RestoreV2_43/_44 = _Send[T=DT_FLOAT, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_304_save/RestoreV2_43", _device="/job:localhost/replica:0/task:0/device:CPU:0"](save/RestoreV2_43)]]' (TF Loading Exception)