WEBVTT Kind: captions Language: en 00:00:04.540 --> 00:00:07.150 align:start position:0% hey<00:00:05.540> everyone<00:00:05.840> I<00:00:06.080> figured<00:00:06.680> that<00:00:06.830> starter 00:00:07.150 --> 00:00:07.160 align:start position:0% hey everyone I figured that starter 00:00:07.160 --> 00:00:08.890 align:start position:0% hey everyone I figured that starter minutes<00:00:07.400> longer<00:00:07.790> I<00:00:08.000> have<00:00:08.240> one<00:00:08.420> more<00:00:08.600> minutes 00:00:08.890 --> 00:00:08.900 align:start position:0% minutes longer I have one more minutes 00:00:08.900 --> 00:00:12.430 align:start position:0% minutes longer I have one more minutes yes<00:00:09.580> so<00:00:10.870> every<00:00:11.870> now<00:00:11.990> and<00:00:12.200> again<00:00:12.230> somebody 00:00:12.430 --> 00:00:12.440 align:start position:0% yes so every now and again somebody 00:00:12.440 --> 00:00:14.740 align:start position:0% yes so every now and again somebody sends<00:00:13.040> me<00:00:13.070> a<00:00:13.160> message<00:00:13.580> on<00:00:13.910> messenger<00:00:14.480> or 00:00:14.740 --> 00:00:14.750 align:start position:0% sends me a message on messenger or 00:00:14.750 --> 00:00:16.870 align:start position:0% sends me a message on messenger or wherever<00:00:14.960> and<00:00:15.410> if<00:00:16.160> message<00:00:16.430> kind<00:00:16.610> of<00:00:16.730> goes 00:00:16.870 --> 00:00:16.880 align:start position:0% wherever and if message kind of goes 00:00:16.880 --> 00:00:20.439 align:start position:0% wherever and if message kind of goes along<00:00:17.060> how<00:00:17.390> do<00:00:17.450> i<00:00:17.599> D<00:00:17.660> bug<00:00:17.900> death<00:00:18.170> and<00:00:18.730> I<00:00:19.730> always 00:00:20.439 --> 00:00:20.449 align:start position:0% along how do i D bug death and I always 00:00:20.449 --> 00:00:22.359 align:start position:0% along how do i D bug death and I always I<00:00:20.780> have<00:00:21.260> a<00:00:21.289> follow-up<00:00:21.650> question<00:00:21.890> what<00:00:22.250> do<00:00:22.310> you 00:00:22.359 --> 00:00:22.369 align:start position:0% I have a follow-up question what do you 00:00:22.369 --> 00:00:23.740 align:start position:0% I have a follow-up question what do you mean<00:00:22.490> debug<00:00:22.730> guests<00:00:23.030> just<00:00:23.300> submit<00:00:23.539> an<00:00:23.660> issue 00:00:23.740 --> 00:00:23.750 align:start position:0% mean debug guests just submit an issue 00:00:23.750 --> 00:00:25.720 align:start position:0% mean debug guests just submit an issue and<00:00:24.110> it<00:00:24.770> turns<00:00:25.039> out<00:00:25.250> that<00:00:25.310> they<00:00:25.699> actually 00:00:25.720 --> 00:00:25.730 align:start position:0% and it turns out that they actually 00:00:25.730 --> 00:00:28.630 align:start position:0% and it turns out that they actually don't<00:00:26.270> have<00:00:26.570> an<00:00:26.690> issue<00:00:26.810> rather<00:00:27.800> they<00:00:28.130> are<00:00:28.160> they 00:00:28.630 --> 00:00:28.640 align:start position:0% don't have an issue rather they are they 00:00:28.640 --> 00:00:30.880 align:start position:0% don't have an issue rather they are they want<00:00:28.760> to<00:00:28.940> know<00:00:29.060> how<00:00:29.300> they<00:00:29.539> can<00:00:29.600> know<00:00:30.020> if<00:00:30.320> gas<00:00:30.619> is 00:00:30.880 --> 00:00:30.890 align:start position:0% want to know how they can know if gas is 00:00:30.890 --> 00:00:33.490 align:start position:0% want to know how they can know if gas is actually<00:00:31.100> working<00:00:32.059> or<00:00:32.300> if<00:00:32.660> gas<00:00:32.900> is<00:00:32.930> working 00:00:33.490 --> 00:00:33.500 align:start position:0% actually working or if gas is working 00:00:33.500 --> 00:00:36.729 align:start position:0% actually working or if gas is working properly<00:00:33.830> or<00:00:34.309> how<00:00:34.460> healthy<00:00:34.879> it<00:00:34.940> is<00:00:35.120> and<00:00:35.739> then 00:00:36.729 --> 00:00:36.739 align:start position:0% properly or how healthy it is and then 00:00:36.739 --> 00:00:38.860 align:start position:0% properly or how healthy it is and then usually<00:00:36.980> what<00:00:37.340> we<00:00:37.460> refer<00:00:37.760> people<00:00:37.790> to<00:00:38.210> is<00:00:38.600> well 00:00:38.860 --> 00:00:38.870 align:start position:0% usually what we refer people to is well 00:00:38.870 --> 00:00:41.229 align:start position:0% usually what we refer people to is well just<00:00:39.380> look<00:00:39.770> at<00:00:39.980> the<00:00:40.100> logs<00:00:40.309> how<00:00:40.700> hard<00:00:40.970> can<00:00:41.180> that 00:00:41.229 --> 00:00:41.239 align:start position:0% just look at the logs how hard can that 00:00:41.239 --> 00:00:41.860 align:start position:0% just look at the logs how hard can that be 00:00:41.860 --> 00:00:41.870 align:start position:0% be 00:00:41.870 --> 00:00:44.680 align:start position:0% be okay<00:00:42.530> I'm<00:00:42.590> being<00:00:42.739> a<00:00:42.860> bit<00:00:42.980> ironic<00:00:43.130> here<00:00:43.579> but<00:00:44.180> the 00:00:44.680 --> 00:00:44.690 align:start position:0% okay I'm being a bit ironic here but the 00:00:44.690 --> 00:00:48.040 align:start position:0% okay I'm being a bit ironic here but the problem<00:00:45.050> with<00:00:45.170> looking<00:00:45.260> at<00:00:45.530> the<00:00:45.620> logs<00:00:45.829> is<00:00:47.050> that 00:00:48.040 --> 00:00:48.050 align:start position:0% problem with looking at the logs is that 00:00:48.050 --> 00:00:51.160 align:start position:0% problem with looking at the logs is that we<00:00:48.440> either<00:00:48.760> arrive<00:00:49.760> in<00:00:49.910> this<00:00:50.059> situation<00:00:50.719> where 00:00:51.160 --> 00:00:51.170 align:start position:0% we either arrive in this situation where 00:00:51.170 --> 00:00:53.470 align:start position:0% we either arrive in this situation where death<00:00:51.860> is<00:00:52.010> just<00:00:52.040> silent<00:00:52.670> and<00:00:52.910> there<00:00:53.059> are<00:00:53.210> no 00:00:53.470 --> 00:00:53.480 align:start position:0% death is just silent and there are no 00:00:53.480 --> 00:00:55.450 align:start position:0% death is just silent and there are no logs<00:00:53.750> and<00:00:54.079> you're<00:00:54.200> just<00:00:54.410> waiting<00:00:54.649> and<00:00:55.219> waiting 00:00:55.450 --> 00:00:55.460 align:start position:0% logs and you're just waiting and waiting 00:00:55.460 --> 00:00:58.119 align:start position:0% logs and you're just waiting and waiting or<00:00:56.450> maybe<00:00:56.629> there's<00:00:56.899> a<00:00:56.989> log<00:00:57.140> every<00:00:57.379> 15<00:00:57.739> seconds 00:00:58.119 --> 00:00:58.129 align:start position:0% or maybe there's a log every 15 seconds 00:00:58.129 --> 00:00:59.410 align:start position:0% or maybe there's a log every 15 seconds but<00:00:58.250> you<00:00:58.370> still<00:00:58.610> have<00:00:58.760> no<00:00:58.910> idea<00:00:58.940> whether 00:00:59.410 --> 00:00:59.420 align:start position:0% but you still have no idea whether 00:00:59.420 --> 00:01:02.709 align:start position:0% but you still have no idea whether that's<00:01:00.079> good<00:01:00.320> or<00:01:00.410> bad<00:01:00.469> or<00:01:01.059> we<00:01:02.059> have<00:01:02.239> the<00:01:02.480> more 00:01:02.709 --> 00:01:02.719 align:start position:0% that's good or bad or we have the more 00:01:02.719 --> 00:01:04.689 align:start position:0% that's good or bad or we have the more ingenious<00:01:03.079> people<00:01:03.559> who<00:01:03.710> raise<00:01:03.980> the<00:01:04.160> log<00:01:04.339> level 00:01:04.689 --> 00:01:04.699 align:start position:0% ingenious people who raise the log level 00:01:04.699 --> 00:01:07.270 align:start position:0% ingenious people who raise the log level and<00:01:05.110> they<00:01:06.110> still<00:01:06.410> have<00:01:06.560> no<00:01:06.680> idea<00:01:06.979> what's<00:01:07.130> going 00:01:07.270 --> 00:01:07.280 align:start position:0% and they still have no idea what's going 00:01:07.280 --> 00:01:10.840 align:start position:0% and they still have no idea what's going on<00:01:07.990> so<00:01:08.990> the<00:01:09.140> problem<00:01:09.560> is<00:01:09.709> that<00:01:09.770> logs<00:01:10.399> are 00:01:10.840 --> 00:01:10.850 align:start position:0% on so the problem is that logs are 00:01:10.850 --> 00:01:12.940 align:start position:0% on so the problem is that logs are really<00:01:11.750> really<00:01:11.990> useful<00:01:12.020> if<00:01:12.590> you<00:01:12.770> know 00:01:12.940 --> 00:01:12.950 align:start position:0% really really useful if you know 00:01:12.950 --> 00:01:14.499 align:start position:0% really really useful if you know specifically<00:01:13.759> something<00:01:14.119> that<00:01:14.360> you're 00:01:14.499 --> 00:01:14.509 align:start position:0% specifically something that you're 00:01:14.509 --> 00:01:16.719 align:start position:0% specifically something that you're looking<00:01:14.810> for<00:01:15.080> but<00:01:15.860> if<00:01:16.009> you<00:01:16.160> just<00:01:16.340> want<00:01:16.550> to<00:01:16.580> get 00:01:16.719 --> 00:01:16.729 align:start position:0% looking for but if you just want to get 00:01:16.729 --> 00:01:18.580 align:start position:0% looking for but if you just want to get a<00:01:16.759> general<00:01:17.119> overview<00:01:17.509> of<00:01:17.720> what<00:01:17.959> your<00:01:18.289> node<00:01:18.440> is 00:01:18.580 --> 00:01:18.590 align:start position:0% a general overview of what your node is 00:01:18.590 --> 00:01:23.969 align:start position:0% a general overview of what your node is doing<00:01:18.800> then<00:01:19.550> logs<00:01:20.479> are<00:01:20.750> perfectly<00:01:22.240> not<00:01:23.240> useful 00:01:23.969 --> 00:01:23.979 align:start position:0% doing then logs are perfectly not useful 00:01:23.979 --> 00:01:28.630 align:start position:0% doing then logs are perfectly not useful so<00:01:24.979> what<00:01:25.190> can<00:01:25.369> we<00:01:25.580> do<00:01:26.060> instead<00:01:26.770> well<00:01:27.770> first 00:01:28.630 --> 00:01:28.640 align:start position:0% so what can we do instead well first 00:01:28.640 --> 00:01:29.609 align:start position:0% so what can we do instead well first things<00:01:28.849> first 00:01:29.609 --> 00:01:29.619 align:start position:0% things first 00:01:29.619 --> 00:01:31.930 align:start position:0% things first people<00:01:30.619> mostly<00:01:30.979> are<00:01:31.069> interested<00:01:31.640> in<00:01:31.759> whether 00:01:31.930 --> 00:01:31.940 align:start position:0% people mostly are interested in whether 00:01:31.940 --> 00:01:34.120 align:start position:0% people mostly are interested in whether they<00:01:32.179> know<00:01:32.390> this<00:01:32.569> working<00:01:32.990> or<00:01:33.140> not<00:01:33.200> and<00:01:33.649> this 00:01:34.120 --> 00:01:34.130 align:start position:0% they know this working or not and this 00:01:34.130 --> 00:01:36.160 align:start position:0% they know this working or not and this is<00:01:34.190> not<00:01:34.429> such<00:01:34.670> a<00:01:34.700> simple<00:01:35.119> question<00:01:35.660> to<00:01:36.020> answer 00:01:36.160 --> 00:01:36.170 align:start position:0% is not such a simple question to answer 00:01:36.170 --> 00:01:38.740 align:start position:0% is not such a simple question to answer because<00:01:36.500> it's<00:01:36.770> not<00:01:36.920> a<00:01:36.950> binary<00:01:37.240> of<00:01:38.240> course<00:01:38.599> if 00:01:38.740 --> 00:01:38.750 align:start position:0% because it's not a binary of course if 00:01:38.750 --> 00:01:40.419 align:start position:0% because it's not a binary of course if they<00:01:38.840> know<00:01:39.020> is<00:01:39.170> not<00:01:39.319> working<00:01:39.770> we<00:01:39.920> see<00:01:40.099> it<00:01:40.160> that 00:01:40.419 --> 00:01:40.429 align:start position:0% they know is not working we see it that 00:01:40.429 --> 00:01:42.100 align:start position:0% they know is not working we see it that it<00:01:40.580> is<00:01:40.729> not<00:01:40.880> working<00:01:41.270> but<00:01:41.390> if<00:01:41.539> it's<00:01:41.660> working 00:01:42.100 --> 00:01:42.110 align:start position:0% it is not working but if it's working 00:01:42.110 --> 00:01:43.900 align:start position:0% it is not working but if it's working then<00:01:42.289> that's<00:01:42.530> an<00:01:42.649> entire<00:01:42.950> spectrum<00:01:43.429> it<00:01:43.700> might 00:01:43.900 --> 00:01:43.910 align:start position:0% then that's an entire spectrum it might 00:01:43.910 --> 00:01:46.120 align:start position:0% then that's an entire spectrum it might be<00:01:44.060> working<00:01:44.270> very<00:01:44.660> well<00:01:44.750> it<00:01:45.470> might<00:01:45.740> be<00:01:45.920> working 00:01:46.120 --> 00:01:46.130 align:start position:0% be working very well it might be working 00:01:46.130 --> 00:01:49.389 align:start position:0% be working very well it might be working just<00:01:46.459> barely<00:01:46.789> but<00:01:47.479> it's<00:01:48.289> still<00:01:48.530> working<00:01:48.770> now 00:01:49.389 --> 00:01:49.399 align:start position:0% just barely but it's still working now 00:01:49.399 --> 00:01:50.680 align:start position:0% just barely but it's still working now the<00:01:49.459> issue<00:01:49.700> is<00:01:49.970> that<00:01:50.000> if<00:01:50.239> it's<00:01:50.360> working<00:01:50.660> very 00:01:50.680 --> 00:01:50.690 align:start position:0% the issue is that if it's working very 00:01:50.690 --> 00:01:52.900 align:start position:0% the issue is that if it's working very well<00:01:50.959> then<00:01:51.349> everything<00:01:51.709> is<00:01:51.830> fine<00:01:52.039> if<00:01:52.310> it's 00:01:52.900 --> 00:01:52.910 align:start position:0% well then everything is fine if it's 00:01:52.910 --> 00:01:55.359 align:start position:0% well then everything is fine if it's just<00:01:53.259> working<00:01:54.259> barely<00:01:54.890> then<00:01:55.099> you<00:01:55.190> might<00:01:55.340> have 00:01:55.359 --> 00:01:55.369 align:start position:0% just working barely then you might have 00:01:55.369 --> 00:01:57.279 align:start position:0% just working barely then you might have an<00:01:55.729> issue<00:01:55.849> at<00:01:56.119> 3:00<00:01:56.360> a.m.<00:01:56.479> in<00:01:56.660> the<00:01:56.690> morning<00:01:57.110> and 00:01:57.279 --> 00:01:57.289 align:start position:0% an issue at 3:00 a.m. in the morning and 00:01:57.289 --> 00:01:59.169 align:start position:0% an issue at 3:00 a.m. in the morning and you<00:01:57.349> want<00:01:57.560> to<00:01:57.679> be<00:01:57.770> able<00:01:57.830> to<00:01:58.009> tell<00:01:58.310> how<00:01:58.729> healthy 00:01:59.169 --> 00:01:59.179 align:start position:0% you want to be able to tell how healthy 00:01:59.179 --> 00:02:01.779 align:start position:0% you want to be able to tell how healthy is<00:01:59.330> your<00:01:59.569> node<00:01:59.750> now<00:02:00.739> it's<00:02:01.310> not<00:02:01.459> suppose<00:02:01.759> that 00:02:01.779 --> 00:02:01.789 align:start position:0% is your node now it's not suppose that 00:02:01.789 --> 00:02:04.660 align:start position:0% is your node now it's not suppose that we<00:02:02.090> start<00:02:02.330> with<00:02:02.649> latest<00:02:03.649> release<00:02:03.890> it<00:02:04.190> kind<00:02:04.489> of 00:02:04.660 --> 00:02:04.670 align:start position:0% we start with latest release it kind of 00:02:04.670 --> 00:02:07.719 align:start position:0% we start with latest release it kind of works<00:02:04.940> but<00:02:05.569> a<00:02:05.630> lot<00:02:05.840> of<00:02:05.869> him<00:02:06.050> flood<00:02:06.739> of<00:02:06.890> things 00:02:07.719 --> 00:02:07.729 align:start position:0% works but a lot of him flood of things 00:02:07.729 --> 00:02:10.510 align:start position:0% works but a lot of him flood of things can<00:02:07.970> actually<00:02:08.390> influence<00:02:08.739> your<00:02:09.739> node<00:02:09.950> in<00:02:10.250> some 00:02:10.510 --> 00:02:10.520 align:start position:0% can actually influence your node in some 00:02:10.520 --> 00:02:12.130 align:start position:0% can actually influence your node in some one<00:02:10.819> direction<00:02:11.239> or<00:02:11.330> the<00:02:11.360> other<00:02:11.510> for<00:02:11.810> example 00:02:12.130 --> 00:02:12.140 align:start position:0% one direction or the other for example 00:02:12.140 --> 00:02:13.589 align:start position:0% one direction or the other for example every<00:02:12.319> time<00:02:12.349> we<00:02:12.620> do<00:02:12.769> a<00:02:12.799> bug-fix<00:02:13.219> release 00:02:13.589 --> 00:02:13.599 align:start position:0% every time we do a bug-fix release 00:02:13.599 --> 00:02:16.020 align:start position:0% every time we do a bug-fix release hopefully<00:02:14.599> it<00:02:14.719> improves<00:02:15.110> the<00:02:15.230> situation<00:02:15.439> and 00:02:16.020 --> 00:02:16.030 align:start position:0% hopefully it improves the situation and 00:02:16.030 --> 00:02:18.220 align:start position:0% hopefully it improves the situation and of<00:02:17.030> course<00:02:17.269> if<00:02:17.480> you<00:02:17.659> run<00:02:17.870> it<00:02:17.989> on 00:02:18.220 --> 00:02:18.230 align:start position:0% of course if you run it on 00:02:18.230 --> 00:02:20.410 align:start position:0% of course if you run it on or<00:02:18.260> powerful<00:02:18.739> hardware<00:02:19.129> that<00:02:19.280> also<00:02:19.730> papers 00:02:20.410 --> 00:02:20.420 align:start position:0% or powerful hardware that also papers 00:02:20.420 --> 00:02:22.589 align:start position:0% or powerful hardware that also papers over<00:02:20.629> any<00:02:20.930> issues<00:02:21.500> that<00:02:21.530> we<00:02:21.769> might<00:02:21.950> have<00:02:22.190> with 00:02:22.589 --> 00:02:22.599 align:start position:0% over any issues that we might have with 00:02:22.599 --> 00:02:25.860 align:start position:0% over any issues that we might have with flaky<00:02:23.599> algorithms<00:02:24.170> or<00:02:24.319> suboptimal<00:02:24.650> things 00:02:25.860 --> 00:02:25.870 align:start position:0% flaky algorithms or suboptimal things 00:02:25.870 --> 00:02:27.699 align:start position:0% flaky algorithms or suboptimal things Network<00:02:26.870> an<00:02:27.019> activity<00:02:27.470> of<00:02:27.590> other 00:02:27.699 --> 00:02:27.709 align:start position:0% Network an activity of other 00:02:27.709 --> 00:02:29.680 align:start position:0% Network an activity of other externalities<00:02:28.430> like<00:02:28.610> workload<00:02:29.150> again 00:02:29.680 --> 00:02:29.690 align:start position:0% externalities like workload again 00:02:29.690 --> 00:02:32.320 align:start position:0% externalities like workload again influency<00:02:30.230> opera<00:02:30.530> up<00:02:30.920> your<00:02:31.730> operation 00:02:32.320 --> 00:02:32.330 align:start position:0% influency opera up your operation 00:02:32.330 --> 00:02:34.509 align:start position:0% influency opera up your operation allottee<00:02:32.720> or<00:02:32.930> however<00:02:33.409> it<00:02:33.650> is<00:02:33.709> pronounced 00:02:34.509 --> 00:02:34.519 align:start position:0% allottee or however it is pronounced 00:02:34.519 --> 00:02:38.410 align:start position:0% allottee or however it is pronounced sorry<00:02:35.440> essentially<00:02:36.440> of<00:02:36.799> course<00:02:36.860> if<00:02:37.430> you<00:02:37.879> put 00:02:38.410 --> 00:02:38.420 align:start position:0% sorry essentially of course if you put 00:02:38.420 --> 00:02:40.270 align:start position:0% sorry essentially of course if you put if<00:02:38.599> you<00:02:38.750> run<00:02:38.900> a<00:02:39.049> girly<00:02:39.410> node<00:02:39.560> on<00:02:39.739> your<00:02:39.920> notebook 00:02:40.270 --> 00:02:40.280 align:start position:0% if you run a girly node on your notebook 00:02:40.280 --> 00:02:42.250 align:start position:0% if you run a girly node on your notebook then<00:02:40.430> it<00:02:40.519> will<00:02:40.700> perform<00:02:41.540> really<00:02:41.870> really<00:02:42.110> well 00:02:42.250 --> 00:02:42.260 align:start position:0% then it will perform really really well 00:02:42.260 --> 00:02:44.170 align:start position:0% then it will perform really really well whereas<00:02:42.650> if<00:02:42.799> you<00:02:42.950> run<00:02:43.160> may<00:02:43.400> not<00:02:43.640> then<00:02:44.060> you 00:02:44.170 --> 00:02:44.180 align:start position:0% whereas if you run may not then you 00:02:44.180 --> 00:02:46.000 align:start position:0% whereas if you run may not then you might<00:02:44.360> run<00:02:44.569> into<00:02:44.629> troubles<00:02:45.230> these<00:02:45.620> are<00:02:45.799> kind 00:02:46.000 --> 00:02:46.010 align:start position:0% might run into troubles these are kind 00:02:46.010 --> 00:02:48.789 align:start position:0% might run into troubles these are kind of<00:02:46.040> natural<00:02:46.549> now<00:02:47.540> the<00:02:47.989> thing<00:02:48.349> is<00:02:48.530> when<00:02:48.709> you 00:02:48.789 --> 00:02:48.799 align:start position:0% of natural now the thing is when you 00:02:48.799 --> 00:02:49.830 align:start position:0% of natural now the thing is when you start<00:02:49.099> running<00:02:49.250> a<00:02:49.610> production 00:02:49.830 --> 00:02:49.840 align:start position:0% start running a production 00:02:49.840 --> 00:02:52.509 align:start position:0% start running a production infrastructure<00:02:50.840> then<00:02:51.379> just<00:02:52.010> knowing<00:02:52.340> that 00:02:52.509 --> 00:02:52.519 align:start position:0% infrastructure then just knowing that 00:02:52.519 --> 00:02:54.009 align:start position:0% infrastructure then just knowing that you<00:02:52.670> know<00:02:52.700> it<00:02:52.910> works<00:02:53.150> is<00:02:53.329> not<00:02:53.510> really<00:02:53.750> enough 00:02:54.009 --> 00:02:54.019 align:start position:0% you know it works is not really enough 00:02:54.019 --> 00:02:55.720 align:start position:0% you know it works is not really enough you<00:02:54.349> want<00:02:54.470> to<00:02:54.650> know<00:02:54.769> how<00:02:55.040> well<00:02:55.370> it<00:02:55.519> works 00:02:55.720 --> 00:02:55.730 align:start position:0% you want to know how well it works 00:02:55.730 --> 00:02:56.949 align:start position:0% you want to know how well it works because<00:02:56.150> it's<00:02:56.329> kind<00:02:56.510> of<00:02:56.540> a<00:02:56.629> game<00:02:56.750> of 00:02:56.949 --> 00:02:56.959 align:start position:0% because it's kind of a game of 00:02:56.959 --> 00:03:00.699 align:start position:0% because it's kind of a game of trade-offs<00:02:57.829> if<00:02:58.340> you<00:02:59.120> are<00:02:59.329> yes<00:03:00.170> you<00:03:00.200> can<00:03:00.530> put 00:03:00.699 --> 00:03:00.709 align:start position:0% trade-offs if you are yes you can put 00:03:00.709 --> 00:03:02.110 align:start position:0% trade-offs if you are yes you can put the<00:03:00.829> most<00:03:01.010> powerful<00:03:01.400> virtual<00:03:01.849> machine 00:03:02.110 --> 00:03:02.120 align:start position:0% the most powerful virtual machine 00:03:02.120 --> 00:03:03.759 align:start position:0% the most powerful virtual machine available<00:03:02.540> on<00:03:02.629> Amazon<00:03:03.019> underneath<00:03:03.410> it<00:03:03.560> and<00:03:03.680> it 00:03:03.759 --> 00:03:03.769 align:start position:0% available on Amazon underneath it and it 00:03:03.769 --> 00:03:05.349 align:start position:0% available on Amazon underneath it and it will<00:03:03.890> probably<00:03:04.280> run<00:03:04.400> perfectly<00:03:04.970> but<00:03:05.090> it<00:03:05.209> will 00:03:05.349 --> 00:03:05.359 align:start position:0% will probably run perfectly but it will 00:03:05.359 --> 00:03:06.849 align:start position:0% will probably run perfectly but it will cost<00:03:05.540> you<00:03:05.750> a<00:03:05.780> lot<00:03:05.840> or<00:03:06.260> you<00:03:06.560> can<00:03:06.709> put<00:03:06.829> a 00:03:06.849 --> 00:03:06.859 align:start position:0% cost you a lot or you can put a 00:03:06.859 --> 00:03:08.319 align:start position:0% cost you a lot or you can put a Raspberry<00:03:07.280> Pi<00:03:07.340> underneath<00:03:07.819> it<00:03:07.970> and<00:03:08.090> it<00:03:08.150> won't 00:03:08.319 --> 00:03:08.329 align:start position:0% Raspberry Pi underneath it and it won't 00:03:08.329 --> 00:03:10.780 align:start position:0% Raspberry Pi underneath it and it won't run<00:03:08.569> so<00:03:09.319> that's<00:03:09.620> how<00:03:10.010> do<00:03:10.069> we<00:03:10.280> found<00:03:10.519> them<00:03:10.700> in 00:03:10.780 --> 00:03:10.790 align:start position:0% run so that's how do we found them in 00:03:10.790 --> 00:03:13.780 align:start position:0% run so that's how do we found them in find<00:03:11.269> the<00:03:11.329> middle<00:03:11.540> ground<00:03:11.660> and<00:03:12.250> the<00:03:13.250> answer<00:03:13.579> is 00:03:13.780 --> 00:03:13.790 align:start position:0% find the middle ground and the answer is 00:03:13.790 --> 00:03:17.289 align:start position:0% find the middle ground and the answer is actually<00:03:14.150> metrics<00:03:15.140> and<00:03:15.379> monitoring<00:03:15.970> now<00:03:16.970> from 00:03:17.289 --> 00:03:17.299 align:start position:0% actually metrics and monitoring now from 00:03:17.299 --> 00:03:19.059 align:start position:0% actually metrics and monitoring now from the<00:03:17.540> very<00:03:17.750> very<00:03:18.290> abstract<00:03:18.680> perspective 00:03:19.059 --> 00:03:19.069 align:start position:0% the very very abstract perspective 00:03:19.069 --> 00:03:21.460 align:start position:0% the very very abstract perspective metrics<00:03:19.910> and<00:03:20.060> monitoring<00:03:20.420> means<00:03:20.660> measure 00:03:21.460 --> 00:03:21.470 align:start position:0% metrics and monitoring means measure 00:03:21.470 --> 00:03:23.920 align:start position:0% metrics and monitoring means measure everything<00:03:21.980> that<00:03:22.280> you<00:03:22.400> can<00:03:22.609> and<00:03:22.880> visualize<00:03:23.690> it 00:03:23.920 --> 00:03:23.930 align:start position:0% everything that you can and visualize it 00:03:23.930 --> 00:03:26.349 align:start position:0% everything that you can and visualize it and<00:03:24.190> this<00:03:25.190> is<00:03:25.340> not<00:03:25.489> something<00:03:25.940> new<00:03:25.970> in<00:03:26.239> the 00:03:26.349 --> 00:03:26.359 align:start position:0% and this is not something new in the 00:03:26.359 --> 00:03:28.089 align:start position:0% and this is not something new in the theorem<00:03:26.660> ecosystem<00:03:27.230> I<00:03:27.319> mean<00:03:27.500> the<00:03:27.650> stats<00:03:27.889> page 00:03:28.089 --> 00:03:28.099 align:start position:0% theorem ecosystem I mean the stats page 00:03:28.099 --> 00:03:29.559 align:start position:0% theorem ecosystem I mean the stats page was<00:03:28.400> available<00:03:28.639> since<00:03:29.090> forever 00:03:29.559 --> 00:03:29.569 align:start position:0% was available since forever 00:03:29.569 --> 00:03:31.470 align:start position:0% was available since forever and<00:03:29.930> everybody<00:03:30.470> knows<00:03:30.650> its<00:03:30.859> usefulness 00:03:31.470 --> 00:03:31.480 align:start position:0% and everybody knows its usefulness 00:03:31.480 --> 00:03:34.360 align:start position:0% and everybody knows its usefulness however<00:03:32.480> once<00:03:33.380> you<00:03:33.470> you<00:03:33.829> want<00:03:34.099> to<00:03:34.160> measure 00:03:34.360 --> 00:03:34.370 align:start position:0% however once you you want to measure 00:03:34.370 --> 00:03:36.220 align:start position:0% however once you you want to measure something<00:03:34.519> more<00:03:35.120> it's<00:03:35.599> really<00:03:35.870> hard<00:03:36.109> to 00:03:36.220 --> 00:03:36.230 align:start position:0% something more it's really hard to 00:03:36.230 --> 00:03:38.080 align:start position:0% something more it's really hard to figure<00:03:36.380> out<00:03:36.530> okay<00:03:36.889> what<00:03:37.099> exactly<00:03:37.880> do<00:03:38.000> you 00:03:38.080 --> 00:03:38.090 align:start position:0% figure out okay what exactly do you 00:03:38.090 --> 00:03:39.460 align:start position:0% figure out okay what exactly do you measure<00:03:38.239> I<00:03:38.389> mean<00:03:38.569> a<00:03:38.599> software<00:03:39.019> system<00:03:39.139> there 00:03:39.460 --> 00:03:39.470 align:start position:0% measure I mean a software system there 00:03:39.470 --> 00:03:41.349 align:start position:0% measure I mean a software system there are<00:03:39.500> gazillions<00:03:40.040> of<00:03:40.220> things<00:03:40.459> that<00:03:40.639> it<00:03:40.760> does<00:03:40.970> so 00:03:41.349 --> 00:03:41.359 align:start position:0% are gazillions of things that it does so 00:03:41.359 --> 00:03:43.120 align:start position:0% are gazillions of things that it does so what<00:03:42.049> is<00:03:42.169> important<00:03:42.680> and<00:03:42.799> what<00:03:42.950> is<00:03:42.980> not 00:03:43.120 --> 00:03:43.130 align:start position:0% what is important and what is not 00:03:43.130 --> 00:03:45.460 align:start position:0% what is important and what is not important<00:03:43.970> and<00:03:44.239> to<00:03:44.900> answer<00:03:45.049> that<00:03:45.230> you 00:03:45.460 --> 00:03:45.470 align:start position:0% important and to answer that you 00:03:45.470 --> 00:03:46.930 align:start position:0% important and to answer that you actually<00:03:45.799> need<00:03:45.950> to<00:03:45.980> have<00:03:46.099> specific<00:03:46.790> questions 00:03:46.930 --> 00:03:46.940 align:start position:0% actually need to have specific questions 00:03:46.940 --> 00:03:48.640 align:start position:0% actually need to have specific questions what<00:03:47.480> kind<00:03:47.510> of<00:03:47.750> questions<00:03:48.200> would<00:03:48.349> you<00:03:48.410> like 00:03:48.640 --> 00:03:48.650 align:start position:0% what kind of questions would you like 00:03:48.650 --> 00:03:51.069 align:start position:0% what kind of questions would you like the<00:03:48.769> answer<00:03:49.160> to<00:03:49.400> and<00:03:49.639> for<00:03:50.269> example<00:03:50.690> us<00:03:50.810> in<00:03:50.989> the 00:03:51.069 --> 00:03:51.079 align:start position:0% the answer to and for example us in the 00:03:51.079 --> 00:03:52.330 align:start position:0% the answer to and for example us in the gasps<00:03:51.260> theme<00:03:51.500> we<00:03:51.620> usually<00:03:51.919> have<00:03:52.069> three 00:03:52.330 --> 00:03:52.340 align:start position:0% gasps theme we usually have three 00:03:52.340 --> 00:03:55.089 align:start position:0% gasps theme we usually have three questions<00:03:52.660> one<00:03:53.660> of<00:03:53.690> them<00:03:53.959> is<00:03:54.109> how<00:03:54.650> do<00:03:54.709> the<00:03:54.919> nose 00:03:55.089 --> 00:03:55.099 align:start position:0% questions one of them is how do the nose 00:03:55.099 --> 00:03:57.400 align:start position:0% questions one of them is how do the nose behave<00:03:55.519> across<00:03:55.819> the<00:03:55.970> globe<00:03:56.209> the<00:03:56.930> second<00:03:57.290> is 00:03:57.400 --> 00:03:57.410 align:start position:0% behave across the globe the second is 00:03:57.410 --> 00:04:00.400 align:start position:0% behave across the globe the second is that<00:03:57.590> if<00:03:57.799> some<00:03:58.160> node<00:03:58.370> looks<00:03:59.269> weird<00:03:59.720> then<00:04:00.230> what 00:04:00.400 --> 00:04:00.410 align:start position:0% that if some node looks weird then what 00:04:00.410 --> 00:04:03.069 align:start position:0% that if some node looks weird then what is<00:04:00.590> it<00:04:00.799> doing<00:04:01.010> actually<00:04:01.639> and<00:04:01.849> last<00:04:02.780> but<00:04:02.959> not 00:04:03.069 --> 00:04:03.079 align:start position:0% is it doing actually and last but not 00:04:03.079 --> 00:04:04.690 align:start position:0% is it doing actually and last but not least<00:04:03.349> if<00:04:03.560> you<00:04:03.650> found<00:04:03.859> an<00:04:04.069> anomaly<00:04:04.250> and<00:04:04.639> we 00:04:04.690 --> 00:04:04.700 align:start position:0% least if you found an anomaly and we 00:04:04.700 --> 00:04:06.520 align:start position:0% least if you found an anomaly and we fixed<00:04:05.030> it<00:04:05.120> then<00:04:05.450> it<00:04:05.599> is<00:04:05.750> really<00:04:05.989> important<00:04:06.410> for 00:04:06.520 --> 00:04:06.530 align:start position:0% fixed it then it is really important for 00:04:06.530 --> 00:04:08.559 align:start position:0% fixed it then it is really important for us<00:04:06.620> to<00:04:06.680> see<00:04:06.980> that<00:04:07.010> okay<00:04:07.310> the<00:04:07.849> old<00:04:07.940> version<00:04:08.359> did 00:04:08.559 --> 00:04:08.569 align:start position:0% us to see that okay the old version did 00:04:08.569 --> 00:04:10.569 align:start position:0% us to see that okay the old version did something<00:04:08.959> weird<00:04:09.230> does<00:04:09.769> the<00:04:09.919> new<00:04:10.010> version<00:04:10.340> fix 00:04:10.569 --> 00:04:10.579 align:start position:0% something weird does the new version fix 00:04:10.579 --> 00:04:12.970 align:start position:0% something weird does the new version fix it<00:04:10.730> yes<00:04:11.000> and<00:04:11.180> does<00:04:11.660> the<00:04:11.780> new<00:04:11.900> version<00:04:12.290> really 00:04:12.970 --> 00:04:12.980 align:start position:0% it yes and does the new version really 00:04:12.980 --> 00:04:16.300 align:start position:0% it yes and does the new version really not<00:04:13.280> ruin<00:04:13.549> anything<00:04:13.760> else<00:04:14.090> yes<00:04:14.389> or<00:04:14.569> no<00:04:14.799> so<00:04:15.799> if 00:04:16.300 --> 00:04:16.310 align:start position:0% not ruin anything else yes or no so if 00:04:16.310 --> 00:04:19.210 align:start position:0% not ruin anything else yes or no so if we<00:04:16.519> dive<00:04:16.699> into<00:04:17.000> actual<00:04:17.709> things<00:04:18.709> actual 00:04:19.210 --> 00:04:19.220 align:start position:0% we dive into actual things actual 00:04:19.220 --> 00:04:21.670 align:start position:0% we dive into actual things actual examples<00:04:19.810> the<00:04:20.810> caffeine<00:04:21.139> is<00:04:21.229> kind<00:04:21.440> of<00:04:21.470> running 00:04:21.670 --> 00:04:21.680 align:start position:0% examples the caffeine is kind of running 00:04:21.680 --> 00:04:23.800 align:start position:0% examples the caffeine is kind of running eight<00:04:22.039> boot<00:04:22.280> notes<00:04:22.550> globally<00:04:23.060> across<00:04:23.240> various 00:04:23.800 --> 00:04:23.810 align:start position:0% eight boot notes globally across various 00:04:23.810 --> 00:04:26.620 align:start position:0% eight boot notes globally across various continents<00:04:24.139> now<00:04:25.060> first<00:04:26.060> question<00:04:26.419> that<00:04:26.450> we 00:04:26.620 --> 00:04:26.630 align:start position:0% continents now first question that we 00:04:26.630 --> 00:04:29.170 align:start position:0% continents now first question that we want<00:04:26.780> to<00:04:26.840> answer<00:04:27.050> is<00:04:27.440> are<00:04:27.890> these<00:04:28.760> boot<00:04:28.969> notes 00:04:29.170 --> 00:04:29.180 align:start position:0% want to answer is are these boot notes 00:04:29.180 --> 00:04:32.020 align:start position:0% want to answer is are these boot notes healthy<00:04:29.599> and<00:04:29.810> how<00:04:30.590> what<00:04:30.979> is<00:04:31.099> the<00:04:31.250> cost 00:04:32.020 --> 00:04:32.030 align:start position:0% healthy and how what is the cost 00:04:32.030 --> 00:04:33.760 align:start position:0% healthy and how what is the cost now<00:04:32.180> whether<00:04:32.840> the<00:04:33.020> blue<00:04:33.140> tones<00:04:33.290> are<00:04:33.440> healthy 00:04:33.760 --> 00:04:33.770 align:start position:0% now whether the blue tones are healthy 00:04:33.770 --> 00:04:35.230 align:start position:0% now whether the blue tones are healthy or<00:04:33.860> not<00:04:34.250> for<00:04:34.550> us<00:04:34.610> it's<00:04:34.820> really<00:04:34.880> simple 00:04:35.230 --> 00:04:35.240 align:start position:0% or not for us it's really simple 00:04:35.240 --> 00:04:37.060 align:start position:0% or not for us it's really simple ordering<00:04:35.690> are<00:04:36.020> they<00:04:36.320> in<00:04:36.500> sync<00:04:36.770> with<00:04:36.950> the 00:04:37.060 --> 00:04:37.070 align:start position:0% ordering are they in sync with the 00:04:37.070 --> 00:04:38.980 align:start position:0% ordering are they in sync with the network<00:04:37.100> or<00:04:37.550> not<00:04:37.700> so<00:04:38.300> we<00:04:38.450> simply<00:04:38.750> just 00:04:38.980 --> 00:04:38.990 align:start position:0% network or not so we simply just 00:04:38.990 --> 00:04:41.320 align:start position:0% network or not so we simply just visualize<00:04:39.530> how<00:04:40.070> many<00:04:40.280> blocks<00:04:40.610> behind<00:04:40.880> there 00:04:41.320 --> 00:04:41.330 align:start position:0% visualize how many blocks behind there 00:04:41.330 --> 00:04:43.420 align:start position:0% visualize how many blocks behind there are<00:04:41.480> from<00:04:41.780> the<00:04:41.900> chain<00:04:42.140> head<00:04:42.380> and<00:04:42.950> immediately 00:04:43.420 --> 00:04:43.430 align:start position:0% are from the chain head and immediately 00:04:43.430 --> 00:04:45.220 align:start position:0% are from the chain head and immediately if<00:04:43.580> it<00:04:43.730> some<00:04:44.060> Boonton<00:04:44.390> falls<00:04:44.630> behind<00:04:44.990> that<00:04:45.110> we 00:04:45.220 --> 00:04:45.230 align:start position:0% if it some Boonton falls behind that we 00:04:45.230 --> 00:04:47.380 align:start position:0% if it some Boonton falls behind that we know<00:04:45.380> that<00:04:45.530> something<00:04:45.830> is<00:04:46.070> wrong<00:04:46.340> but<00:04:46.610> this 00:04:47.380 --> 00:04:47.390 align:start position:0% know that something is wrong but this 00:04:47.390 --> 00:04:49.090 align:start position:0% know that something is wrong but this just<00:04:47.600> gives<00:04:47.810> us<00:04:47.960> the<00:04:48.080> binary<00:04:48.440> thing<00:04:48.740> we<00:04:48.950> know 00:04:49.090 --> 00:04:49.100 align:start position:0% just gives us the binary thing we know 00:04:49.100 --> 00:04:50.830 align:start position:0% just gives us the binary thing we know that<00:04:49.310> is<00:04:49.430> healthy<00:04:49.760> or<00:04:49.790> not<00:04:50.000> but<00:04:50.270> we<00:04:50.390> don't<00:04:50.600> know 00:04:50.830 --> 00:04:50.840 align:start position:0% that is healthy or not but we don't know 00:04:50.840 --> 00:04:53.380 align:start position:0% that is healthy or not but we don't know what's<00:04:51.380> the<00:04:51.590> cost<00:04:51.830> or<00:04:52.100> how<00:04:52.250> healthy<00:04:52.700> it<00:04:52.760> is<00:04:52.910> now 00:04:53.380 --> 00:04:53.390 align:start position:0% what's the cost or how healthy it is now 00:04:53.390 --> 00:04:55.840 align:start position:0% what's the cost or how healthy it is now if<00:04:53.570> we<00:04:53.780> if<00:04:54.710> in<00:04:54.920> order<00:04:55.130> to<00:04:55.220> find<00:04:55.460> out<00:04:55.610> actually 00:04:55.840 --> 00:04:55.850 align:start position:0% if we if in order to find out actually 00:04:55.850 --> 00:04:57.760 align:start position:0% if we if in order to find out actually how<00:04:56.060> healthy<00:04:56.690> it<00:04:56.750> is<00:04:56.870> we<00:04:57.110> actually<00:04:57.140> need<00:04:57.650> to 00:04:57.760 --> 00:04:57.770 align:start position:0% how healthy it is we actually need to 00:04:57.770 --> 00:05:00.100 align:start position:0% how healthy it is we actually need to also<00:04:57.950> chart<00:04:58.700> out<00:04:58.910> the<00:04:59.240> resource<00:04:59.660> consumption 00:05:00.100 --> 00:05:00.110 align:start position:0% also chart out the resource consumption 00:05:00.110 --> 00:05:02.320 align:start position:0% also chart out the resource consumption of<00:05:00.350> it<00:05:00.500> and<00:05:00.740> as<00:05:01.130> a<00:05:01.280> computing<00:05:01.730> system<00:05:02.060> you<00:05:02.150> have 00:05:02.320 --> 00:05:02.330 align:start position:0% of it and as a computing system you have 00:05:02.330 --> 00:05:05.650 align:start position:0% of it and as a computing system you have four<00:05:02.570> major<00:05:02.750> resources<00:05:03.370> CPU<00:05:04.370> memory<00:05:04.550> disk<00:05:05.000> and 00:05:05.650 --> 00:05:05.660 align:start position:0% four major resources CPU memory disk and 00:05:05.660 --> 00:05:07.870 align:start position:0% four major resources CPU memory disk and network<00:05:05.800> and<00:05:06.800> of<00:05:07.010> course<00:05:07.220> you<00:05:07.310> can<00:05:07.430> split<00:05:07.670> this 00:05:07.870 --> 00:05:07.880 align:start position:0% network and of course you can split this 00:05:07.880 --> 00:05:09.400 align:start position:0% network and of course you can split this cup<00:05:08.090> a<00:05:08.210> bit<00:05:08.360> and<00:05:08.540> network<00:05:08.840> up<00:05:08.990> a<00:05:09.080> bit<00:05:09.260> in 00:05:09.400 --> 00:05:09.410 align:start position:0% cup a bit and network up a bit in 00:05:09.410 --> 00:05:12.580 align:start position:0% cup a bit and network up a bit in input/output<00:05:10.190> etc<00:05:10.730> but<00:05:11.510> only<00:05:11.810> know<00:05:11.960> if<00:05:12.260> we 00:05:12.580 --> 00:05:12.590 align:start position:0% input/output etc but only know if we 00:05:12.590 --> 00:05:15.400 align:start position:0% input/output etc but only know if we chart<00:05:13.490> all<00:05:13.760> of<00:05:14.030> our<00:05:14.180> eight<00:05:14.330> boot<00:05:14.570> nodes<00:05:14.840> across 00:05:15.400 --> 00:05:15.410 align:start position:0% chart all of our eight boot nodes across 00:05:15.410 --> 00:05:17.710 align:start position:0% chart all of our eight boot nodes across these<00:05:15.800> four<00:05:16.160> matrix<00:05:16.550> types<00:05:16.820> then<00:05:17.330> immediately 00:05:17.710 --> 00:05:17.720 align:start position:0% these four matrix types then immediately 00:05:17.720 --> 00:05:20.380 align:start position:0% these four matrix types then immediately we<00:05:18.140> can<00:05:18.170> see<00:05:18.380> anomalies<00:05:19.130> for<00:05:19.310> example<00:05:19.700> on<00:05:19.880> the 00:05:20.380 --> 00:05:20.390 align:start position:0% we can see anomalies for example on the 00:05:20.390 --> 00:05:21.760 align:start position:0% we can see anomalies for example on the memory<00:05:20.660> chart<00:05:20.930> we<00:05:21.050> can<00:05:21.080> see<00:05:21.230> that<00:05:21.530> the<00:05:21.650> yellow 00:05:21.760 --> 00:05:21.770 align:start position:0% memory chart we can see that the yellow 00:05:21.770 --> 00:05:23.950 align:start position:0% memory chart we can see that the yellow boot<00:05:22.100> node<00:05:22.310> is<00:05:22.460> actually<00:05:22.610> using<00:05:23.300> two<00:05:23.540> gigabyte 00:05:23.950 --> 00:05:23.960 align:start position:0% boot node is actually using two gigabyte 00:05:23.960 --> 00:05:27.910 align:start position:0% boot node is actually using two gigabyte more<00:05:24.200> memory<00:05:24.590> than<00:05:24.770> the<00:05:25.070> rest<00:05:25.400> or<00:05:26.530> CPU<00:05:27.530> is<00:05:27.680> a 00:05:27.910 --> 00:05:27.920 align:start position:0% more memory than the rest or CPU is a 00:05:27.920 --> 00:05:29.860 align:start position:0% more memory than the rest or CPU is a similar<00:05:28.520> thing<00:05:28.760> and<00:05:28.940> immediately<00:05:29.750> you<00:05:29.840> have 00:05:29.860 --> 00:05:29.870 align:start position:0% similar thing and immediately you have 00:05:29.870 --> 00:05:32.320 align:start position:0% similar thing and immediately you have something<00:05:30.650> to<00:05:30.920> take<00:05:31.130> care<00:05:31.160> of<00:05:31.520> and<00:05:31.820> the<00:05:32.060> other 00:05:32.320 --> 00:05:32.330 align:start position:0% something to take care of and the other 00:05:32.330 --> 00:05:34.600 align:start position:0% something to take care of and the other interesting<00:05:33.020> aspect<00:05:33.410> of<00:05:33.500> it<00:05:33.680> is<00:05:33.860> that<00:05:34.370> if<00:05:34.490> you 00:05:34.600 --> 00:05:34.610 align:start position:0% interesting aspect of it is that if you 00:05:34.610 --> 00:05:37.690 align:start position:0% interesting aspect of it is that if you know<00:05:34.820> your<00:05:35.420> notes<00:05:35.660> have<00:05:36.100> 13<00:05:37.100> gigabyte<00:05:37.610> of 00:05:37.690 --> 00:05:37.700 align:start position:0% know your notes have 13 gigabyte of 00:05:37.700 --> 00:05:39.670 align:start position:0% know your notes have 13 gigabyte of sorry<00:05:37.910> 32<00:05:38.390> gigabytes<00:05:38.810> of<00:05:38.930> memory<00:05:38.990> and<00:05:39.440> you<00:05:39.590> are 00:05:39.670 --> 00:05:39.680 align:start position:0% sorry 32 gigabytes of memory and you are 00:05:39.680 --> 00:05:41.500 align:start position:0% sorry 32 gigabytes of memory and you are at<00:05:39.770> 8<00:05:40.040> then<00:05:40.520> you<00:05:40.610> don't<00:05:40.820> care<00:05:41.000> about<00:05:41.120> it<00:05:41.390> you're 00:05:41.500 --> 00:05:41.510 align:start position:0% at 8 then you don't care about it you're 00:05:41.510 --> 00:05:43.480 align:start position:0% at 8 then you don't care about it you're really<00:05:41.690> far<00:05:42.020> away<00:05:42.260> from<00:05:42.320> crashing<00:05:43.160> however 00:05:43.480 --> 00:05:43.490 align:start position:0% really far away from crashing however 00:05:43.490 --> 00:05:45.730 align:start position:0% really far away from crashing however once<00:05:44.210> you<00:05:44.360> start<00:05:44.660> reaching<00:05:45.080> your<00:05:45.380> limits 00:05:45.730 --> 00:05:45.740 align:start position:0% once you start reaching your limits 00:05:45.740 --> 00:05:47.470 align:start position:0% once you start reaching your limits that's<00:05:46.400> when<00:05:46.640> things<00:05:46.820> will<00:05:47.030> start<00:05:47.330> going 00:05:47.470 --> 00:05:47.480 align:start position:0% that's when things will start going 00:05:47.480 --> 00:05:49.660 align:start position:0% that's when things will start going wrong<00:05:47.840> so<00:05:48.260> you<00:05:48.320> can<00:05:48.590> just<00:05:49.130> looking<00:05:49.430> at<00:05:49.580> this 00:05:49.660 --> 00:05:49.670 align:start position:0% wrong so you can just looking at this 00:05:49.670 --> 00:05:51.790 align:start position:0% wrong so you can just looking at this dashboard<00:05:50.060> you<00:05:50.390> can<00:05:50.420> immediately<00:05:50.810> see<00:05:51.350> how 00:05:51.790 --> 00:05:51.800 align:start position:0% dashboard you can immediately see how 00:05:51.800 --> 00:05:55.480 align:start position:0% dashboard you can immediately see how close<00:05:52.100> you<00:05:52.340> are<00:05:52.370> to<00:05:52.550> failure<00:05:53.230> cool<00:05:54.230> now 00:05:55.480 --> 00:05:55.490 align:start position:0% close you are to failure cool now 00:05:55.490 --> 00:05:57.870 align:start position:0% close you are to failure cool now just<00:05:56.120> to<00:05:56.300> dive<00:05:56.510> into<00:05:56.720> one<00:05:56.840> of<00:05:57.020> these<00:05:57.140> examples 00:05:57.870 --> 00:05:57.880 align:start position:0% just to dive into one of these examples 00:05:57.880 --> 00:06:00.280 align:start position:0% just to dive into one of these examples DevOps<00:05:58.880> team<00:05:59.000> a<00:05:59.330> couple<00:05:59.510> of<00:05:59.690> weeks<00:05:59.840> ago<00:05:59.990> told 00:06:00.280 --> 00:06:00.290 align:start position:0% DevOps team a couple of weeks ago told 00:06:00.290 --> 00:06:01.930 align:start position:0% DevOps team a couple of weeks ago told us<00:06:00.440> that<00:06:00.650> hey<00:06:00.920> you're<00:06:01.250> kind<00:06:01.460> of<00:06:01.550> using<00:06:01.790> too 00:06:01.930 --> 00:06:01.940 align:start position:0% us that hey you're kind of using too 00:06:01.940 --> 00:06:03.520 align:start position:0% us that hey you're kind of using too much<00:06:02.060> bandwidth<00:06:02.510> I<00:06:02.810> mean<00:06:03.080> it<00:06:03.170> costs<00:06:03.350> too<00:06:03.500> much 00:06:03.520 --> 00:06:03.530 align:start position:0% much bandwidth I mean it costs too much 00:06:03.530 --> 00:06:04.870 align:start position:0% much bandwidth I mean it costs too much they<00:06:03.950> care<00:06:04.160> about<00:06:04.250> the<00:06:04.430> money<00:06:04.580> not<00:06:04.760> the 00:06:04.870 --> 00:06:04.880 align:start position:0% they care about the money not the 00:06:04.880 --> 00:06:07.810 align:start position:0% they care about the money not the bandwidth<00:06:05.210> and<00:06:05.830> so<00:06:06.830> we<00:06:06.950> looked<00:06:07.130> at<00:06:07.250> our<00:06:07.400> charts 00:06:07.810 --> 00:06:07.820 align:start position:0% bandwidth and so we looked at our charts 00:06:07.820 --> 00:06:09.190 align:start position:0% bandwidth and so we looked at our charts and<00:06:07.970> yes<00:06:08.120> indeed<00:06:08.420> the<00:06:08.600> boot<00:06:08.750> nodes<00:06:08.990> are 00:06:09.190 --> 00:06:09.200 align:start position:0% and yes indeed the boot nodes are 00:06:09.200 --> 00:06:12.220 align:start position:0% and yes indeed the boot nodes are actually<00:06:09.680> pushing<00:06:10.430> out<00:06:10.760> five<00:06:11.210> six<00:06:11.240> megabytes 00:06:12.220 --> 00:06:12.230 align:start position:0% actually pushing out five six megabytes 00:06:12.230 --> 00:06:14.920 align:start position:0% actually pushing out five six megabytes per<00:06:12.500> second<00:06:13.010> of<00:06:13.130> data<00:06:13.340> and<00:06:13.850> we're<00:06:14.720> wondering 00:06:14.920 --> 00:06:14.930 align:start position:0% per second of data and we're wondering 00:06:14.930 --> 00:06:18.340 align:start position:0% per second of data and we're wondering okay<00:06:15.440> what<00:06:16.310> does<00:06:16.700> the<00:06:17.030> boot<00:06:17.180> not<00:06:17.330> do<00:06:17.750> I<00:06:17.990> mean 00:06:18.340 --> 00:06:18.350 align:start position:0% okay what does the boot not do I mean 00:06:18.350 --> 00:06:20.380 align:start position:0% okay what does the boot not do I mean this<00:06:18.740> chart<00:06:19.040> tells<00:06:19.250> us<00:06:19.460> how<00:06:19.640> much<00:06:19.790> it<00:06:19.940> costs 00:06:20.380 --> 00:06:20.390 align:start position:0% this chart tells us how much it costs 00:06:20.390 --> 00:06:23.050 align:start position:0% this chart tells us how much it costs our<00:06:20.570> resource<00:06:21.500> wise<00:06:21.740> but<00:06:22.490> we<00:06:22.610> don't<00:06:22.790> know<00:06:22.910> what 00:06:23.050 --> 00:06:23.060 align:start position:0% our resource wise but we don't know what 00:06:23.060 --> 00:06:24.610 align:start position:0% our resource wise but we don't know what it's<00:06:23.210> actually<00:06:23.360> doing<00:06:23.840> with<00:06:23.960> those<00:06:24.140> resources 00:06:24.610 --> 00:06:24.620 align:start position:0% it's actually doing with those resources 00:06:24.620 --> 00:06:26.860 align:start position:0% it's actually doing with those resources so<00:06:25.220> those<00:06:25.370> are<00:06:25.580> actually<00:06:26.030> our<00:06:26.180> intermediate 00:06:26.860 --> 00:06:26.870 align:start position:0% so those are actually our intermediate 00:06:26.870 --> 00:06:29.440 align:start position:0% so those are actually our intermediate level<00:06:27.410> monitoring<00:06:27.890> questions<00:06:28.180> let's<00:06:29.180> try<00:06:29.390> to 00:06:29.440 --> 00:06:29.450 align:start position:0% level monitoring questions let's try to 00:06:29.450 --> 00:06:32.830 align:start position:0% level monitoring questions let's try to figure<00:06:29.690> out<00:06:29.930> what<00:06:30.350> a<00:06:31.060> theorem<00:06:32.060> node<00:06:32.270> is<00:06:32.480> doing 00:06:32.830 --> 00:06:32.840 align:start position:0% figure out what a theorem node is doing 00:06:32.840 --> 00:06:35.020 align:start position:0% figure out what a theorem node is doing with<00:06:33.110> its<00:06:33.350> disk<00:06:34.160> resources<00:06:34.850> or<00:06:35.000> other 00:06:35.020 --> 00:06:35.030 align:start position:0% with its disk resources or other 00:06:35.030 --> 00:06:36.640 align:start position:0% with its disk resources or other resources<00:06:35.690> now<00:06:35.900> in<00:06:35.990> case<00:06:36.140> of<00:06:36.260> not<00:06:36.440> working 00:06:36.640 --> 00:06:36.650 align:start position:0% resources now in case of not working 00:06:36.650 --> 00:06:40.960 align:start position:0% resources now in case of not working what<00:06:37.340> can<00:06:37.550> we<00:06:37.700> do<00:06:37.850> well<00:06:38.630> first<00:06:38.840> up<00:06:39.430> we<00:06:40.430> have<00:06:40.460> on 00:06:40.960 --> 00:06:40.970 align:start position:0% what can we do well first up we have on 00:06:40.970 --> 00:06:42.430 align:start position:0% what can we do well first up we have on previously<00:06:41.480> we've<00:06:41.600> measured<00:06:41.810> the<00:06:41.990> total 00:06:42.430 --> 00:06:42.440 align:start position:0% previously we've measured the total 00:06:42.440 --> 00:06:45.340 align:start position:0% previously we've measured the total bandwidth<00:06:43.090> usage<00:06:44.090> but<00:06:44.720> we<00:06:44.960> would<00:06:45.080> like<00:06:45.170> to 00:06:45.340 --> 00:06:45.350 align:start position:0% bandwidth usage but we would like to 00:06:45.350 --> 00:06:45.939 align:start position:0% bandwidth usage but we would like to also<00:06:45.470> match 00:06:45.939 --> 00:06:45.949 align:start position:0% also match 00:06:45.949 --> 00:06:48.010 align:start position:0% also match the<00:06:46.400> individual<00:06:46.909> map<00:06:47.090> bandwidth<00:06:47.539> usage<00:06:47.689> of 00:06:48.010 --> 00:06:48.020 align:start position:0% the individual map bandwidth usage of 00:06:48.020 --> 00:06:50.170 align:start position:0% the individual map bandwidth usage of individual<00:06:48.530> protocols<00:06:49.129> since<00:06:49.340> the<00:06:49.819> Bouton's 00:06:50.170 --> 00:06:50.180 align:start position:0% individual protocols since the Bouton's 00:06:50.180 --> 00:06:51.999 align:start position:0% individual protocols since the Bouton's are<00:06:50.330> running<00:06:50.689> aetherium<00:06:51.169> and<00:06:51.379> light<00:06:51.650> client 00:06:51.999 --> 00:06:52.009 align:start position:0% are running aetherium and light client 00:06:52.009 --> 00:06:53.619 align:start position:0% are running aetherium and light client protocols<00:06:52.460> those<00:06:52.819> two<00:06:52.999> are<00:06:53.120> the<00:06:53.210> ones<00:06:53.330> we<00:06:53.539> want 00:06:53.619 --> 00:06:53.629 align:start position:0% protocols those two are the ones we want 00:06:53.629 --> 00:06:56.170 align:start position:0% protocols those two are the ones we want to<00:06:53.749> run<00:06:53.990> but<00:06:54.800> we<00:06:54.949> can<00:06:55.099> go<00:06:55.249> even<00:06:55.430> further<00:06:55.610> down 00:06:56.170 --> 00:06:56.180 align:start position:0% to run but we can go even further down 00:06:56.180 --> 00:06:59.469 align:start position:0% to run but we can go even further down and<00:06:56.449> actually<00:06:57.020> measure<00:06:57.199> the<00:06:58.120> network<00:06:59.120> usage 00:06:59.469 --> 00:06:59.479 align:start position:0% and actually measure the network usage 00:06:59.479 --> 00:07:01.600 align:start position:0% and actually measure the network usage of<00:06:59.629> each<00:06:59.840> individual<00:07:00.560> Network<00:07:00.979> packet 00:07:01.600 --> 00:07:01.610 align:start position:0% of each individual Network packet 00:07:01.610 --> 00:07:03.040 align:start position:0% of each individual Network packet I<00:07:01.639> mean<00:07:01.849> packet<00:07:02.150> type<00:07:02.330> within<00:07:02.779> those 00:07:03.040 --> 00:07:03.050 align:start position:0% I mean packet type within those 00:07:03.050 --> 00:07:05.709 align:start position:0% I mean packet type within those protocols<00:07:03.379> and<00:07:03.830> if<00:07:04.370> you<00:07:04.490> chart<00:07:04.759> these<00:07:04.939> we<00:07:05.539> get 00:07:05.709 --> 00:07:05.719 align:start position:0% protocols and if you chart these we get 00:07:05.719 --> 00:07:07.890 align:start position:0% protocols and if you chart these we get these<00:07:05.840> really<00:07:06.319> nice<00:07:06.499> fancy<00:07:06.889> spikes<00:07:07.580> and 00:07:07.890 --> 00:07:07.900 align:start position:0% these really nice fancy spikes and 00:07:07.900 --> 00:07:10.689 align:start position:0% these really nice fancy spikes and immediately<00:07:08.900> we<00:07:09.199> have<00:07:09.409> three<00:07:10.129> things<00:07:10.430> that 00:07:10.689 --> 00:07:10.699 align:start position:0% immediately we have three things that 00:07:10.699 --> 00:07:14.649 align:start position:0% immediately we have three things that are<00:07:10.999> really<00:07:11.240> really<00:07:11.629> obviously<00:07:12.610> strange<00:07:13.659> for 00:07:14.649 --> 00:07:14.659 align:start position:0% are really really obviously strange for 00:07:14.659 --> 00:07:17.230 align:start position:0% are really really obviously strange for example<00:07:15.080> in<00:07:15.319> the<00:07:15.889> top<00:07:16.129> two<00:07:16.370> charts<00:07:16.789> those<00:07:17.060> are 00:07:17.230 --> 00:07:17.240 align:start position:0% example in the top two charts those are 00:07:17.240 --> 00:07:19.570 align:start position:0% example in the top two charts those are the<00:07:17.419> input<00:07:18.199> an<00:07:18.349> upload<00:07:18.949> and<00:07:19.129> download<00:07:19.250> speeds 00:07:19.570 --> 00:07:19.580 align:start position:0% the input an upload and download speeds 00:07:19.580 --> 00:07:21.279 align:start position:0% the input an upload and download speeds of<00:07:19.969> the<00:07:20.000> theorem<00:07:20.449> protocol<00:07:20.870> and<00:07:21.050> you<00:07:21.080> can 00:07:21.279 --> 00:07:21.289 align:start position:0% of the theorem protocol and you can 00:07:21.289 --> 00:07:23.679 align:start position:0% of the theorem protocol and you can immediately<00:07:21.770> see<00:07:22.340> that<00:07:22.580> the<00:07:22.849> light<00:07:23.300> blue 00:07:23.679 --> 00:07:23.689 align:start position:0% immediately see that the light blue 00:07:23.689 --> 00:07:25.959 align:start position:0% immediately see that the light blue thing<00:07:24.349> is<00:07:24.650> causing<00:07:25.099> quite<00:07:25.279> a<00:07:25.370> lot<00:07:25.580> of<00:07:25.610> traffic 00:07:25.959 --> 00:07:25.969 align:start position:0% thing is causing quite a lot of traffic 00:07:25.969 --> 00:07:28.450 align:start position:0% thing is causing quite a lot of traffic and<00:07:26.289> I'm<00:07:27.289> not<00:07:27.529> sure<00:07:27.650> whether<00:07:27.800> it's<00:07:28.009> visible<00:07:28.340> or 00:07:28.450 --> 00:07:28.460 align:start position:0% and I'm not sure whether it's visible or 00:07:28.460 --> 00:07:30.730 align:start position:0% and I'm not sure whether it's visible or not<00:07:28.520> but<00:07:28.909> I<00:07:29.180> will<00:07:29.389> tell<00:07:29.659> you<00:07:29.779> the<00:07:30.319> white<00:07:30.500> blue 00:07:30.730 --> 00:07:30.740 align:start position:0% not but I will tell you the white blue 00:07:30.740 --> 00:07:32.860 align:start position:0% not but I will tell you the white blue has<00:07:30.979> the<00:07:31.129> label<00:07:31.279> of<00:07:31.669> transaction<00:07:32.240> propagation 00:07:32.860 --> 00:07:32.870 align:start position:0% has the label of transaction propagation 00:07:32.870 --> 00:07:35.950 align:start position:0% has the label of transaction propagation so<00:07:33.740> just<00:07:34.669> by<00:07:34.819> looking<00:07:35.120> at<00:07:35.210> these<00:07:35.419> charts<00:07:35.870> you 00:07:35.950 --> 00:07:35.960 align:start position:0% so just by looking at these charts you 00:07:35.960 --> 00:07:37.089 align:start position:0% so just by looking at these charts you can<00:07:36.110> immediately<00:07:36.259> see<00:07:36.680> that<00:07:36.710> there's 00:07:37.089 --> 00:07:37.099 align:start position:0% can immediately see that there's 00:07:37.099 --> 00:07:38.589 align:start position:0% can immediately see that there's something<00:07:37.370> wrong<00:07:37.729> with<00:07:38.060> transaction 00:07:38.589 --> 00:07:38.599 align:start position:0% something wrong with transaction 00:07:38.599 --> 00:07:41.079 align:start position:0% something wrong with transaction propagation<00:07:39.199> it's<00:07:39.979> taking<00:07:40.339> about<00:07:40.550> 1.1 00:07:41.079 --> 00:07:41.089 align:start position:0% propagation it's taking about 1.1 00:07:41.089 --> 00:07:45.579 align:start position:0% propagation it's taking about 1.1 megabyte<00:07:41.599> per<00:07:41.779> second<00:07:42.169> traffic<00:07:42.860> and<00:07:44.589> maybe 00:07:45.579 --> 00:07:45.589 align:start position:0% megabyte per second traffic and maybe 00:07:45.589 --> 00:07:47.110 align:start position:0% megabyte per second traffic and maybe this<00:07:45.830> is<00:07:46.009> enough<00:07:46.189> for<00:07:46.219> you<00:07:46.430> maybe<00:07:46.789> it<00:07:46.939> is<00:07:47.089> not 00:07:47.110 --> 00:07:47.120 align:start position:0% this is enough for you maybe it is not 00:07:47.120 --> 00:07:48.939 align:start position:0% this is enough for you maybe it is not enough<00:07:47.479> for<00:07:47.509> you<00:07:47.810> to<00:07:47.990> decide<00:07:48.379> what's<00:07:48.830> next 00:07:48.939 --> 00:07:48.949 align:start position:0% enough for you to decide what's next 00:07:48.949 --> 00:07:51.610 align:start position:0% enough for you to decide what's next step<00:07:49.370> but<00:07:49.789> in<00:07:49.909> our<00:07:50.060> case<00:07:50.330> yes<00:07:50.900> we<00:07:51.169> are<00:07:51.289> kind<00:07:51.529> of 00:07:51.610 --> 00:07:51.620 align:start position:0% step but in our case yes we are kind of 00:07:51.620 --> 00:07:53.529 align:start position:0% step but in our case yes we are kind of aware<00:07:51.889> of<00:07:52.069> how<00:07:52.279> the<00:07:52.339> transaction<00:07:52.969> propagation 00:07:53.529 --> 00:07:53.539 align:start position:0% aware of how the transaction propagation 00:07:53.539 --> 00:07:55.510 align:start position:0% aware of how the transaction propagation works<00:07:53.779> and<00:07:54.020> this<00:07:54.740> means<00:07:54.919> that<00:07:55.039> we<00:07:55.219> actually 00:07:55.510 --> 00:07:55.520 align:start position:0% works and this means that we actually 00:07:55.520 --> 00:07:57.579 align:start position:0% works and this means that we actually need<00:07:55.699> to<00:07:55.849> roll<00:07:56.000> out<00:07:56.180> a<00:07:56.210> new<00:07:56.419> aetherium<00:07:57.379> sub 00:07:57.579 --> 00:07:57.589 align:start position:0% need to roll out a new aetherium sub 00:07:57.589 --> 00:07:59.050 align:start position:0% need to roll out a new aetherium sub protocol<00:07:58.069> or<00:07:58.219> I<00:07:58.250> mean<00:07:58.370> a<00:07:58.430> new<00:07:58.550> version<00:07:58.789> of<00:07:58.939> the 00:07:59.050 --> 00:07:59.060 align:start position:0% protocol or I mean a new version of the 00:07:59.060 --> 00:08:01.899 align:start position:0% protocol or I mean a new version of the theorem<00:07:59.300> protocol<00:07:59.750> with<00:07:59.870> an<00:07:59.990> alternative<00:08:00.909> way 00:08:01.899 --> 00:08:01.909 align:start position:0% theorem protocol with an alternative way 00:08:01.909 --> 00:08:03.459 align:start position:0% theorem protocol with an alternative way to<00:08:01.939> propagate<00:08:02.330> transactions<00:08:03.169> because 00:08:03.459 --> 00:08:03.469 align:start position:0% to propagate transactions because 00:08:03.469 --> 00:08:04.929 align:start position:0% to propagate transactions because otherwise<00:08:03.710> there's<00:08:04.069> no<00:08:04.219> way<00:08:04.250> to<00:08:04.460> get<00:08:04.759> this 00:08:04.929 --> 00:08:04.939 align:start position:0% otherwise there's no way to get this 00:08:04.939 --> 00:08:06.700 align:start position:0% otherwise there's no way to get this down<00:08:05.180> but<00:08:05.750> we<00:08:05.930> immediately<00:08:06.259> know<00:08:06.500> what<00:08:06.649> the 00:08:06.700 --> 00:08:06.710 align:start position:0% down but we immediately know what the 00:08:06.710 --> 00:08:09.700 align:start position:0% down but we immediately know what the fault<00:08:07.069> is<00:08:07.250> and<00:08:07.460> we<00:08:08.029> have<00:08:08.599> an<00:08:08.750> idea<00:08:09.110> of<00:08:09.349> how<00:08:09.649> to 00:08:09.700 --> 00:08:09.710 align:start position:0% fault is and we have an idea of how to 00:08:09.710 --> 00:08:12.820 align:start position:0% fault is and we have an idea of how to fix<00:08:10.009> it<00:08:10.279> now<00:08:11.270> the<00:08:11.389> other<00:08:11.620> anomalies<00:08:12.620> from 00:08:12.820 --> 00:08:12.830 align:start position:0% fix it now the other anomalies from 00:08:12.830 --> 00:08:15.010 align:start position:0% fix it now the other anomalies from these<00:08:13.039> charts<00:08:13.250> are<00:08:13.580> that<00:08:14.360> the<00:08:14.539> boot<00:08:14.689> nodes<00:08:14.899> are 00:08:15.010 --> 00:08:15.020 align:start position:0% these charts are that the boot nodes are 00:08:15.020 --> 00:08:16.779 align:start position:0% these charts are that the boot nodes are uploading<00:08:15.589> a<00:08:15.680> significant<00:08:16.219> amount<00:08:16.430> of<00:08:16.580> data 00:08:16.779 --> 00:08:16.789 align:start position:0% uploading a significant amount of data 00:08:16.789 --> 00:08:19.540 align:start position:0% uploading a significant amount of data and<00:08:17.270> our<00:08:17.509> guess<00:08:17.810> again<00:08:18.289> is<00:08:18.500> that<00:08:18.979> since<00:08:19.370> these 00:08:19.540 --> 00:08:19.550 align:start position:0% and our guess again is that since these 00:08:19.550 --> 00:08:22.149 align:start position:0% and our guess again is that since these are<00:08:19.729> the<00:08:19.759> boot<00:08:20.029> nodes<00:08:20.860> everybody's<00:08:21.860> trying<00:08:22.099> to 00:08:22.149 --> 00:08:22.159 align:start position:0% are the boot nodes everybody's trying to 00:08:22.159 --> 00:08:24.249 align:start position:0% are the boot nodes everybody's trying to synchronize<00:08:22.699> from<00:08:22.969> them<00:08:23.180> so<00:08:23.750> if<00:08:23.870> you<00:08:23.899> want<00:08:24.080> to 00:08:24.249 --> 00:08:24.259 align:start position:0% synchronize from them so if you want to 00:08:24.259 --> 00:08:26.740 align:start position:0% synchronize from them so if you want to stop<00:08:25.039> people<00:08:25.219> from<00:08:25.399> synchronizing<00:08:26.180> them<00:08:26.330> well 00:08:26.740 --> 00:08:26.750 align:start position:0% stop people from synchronizing them well 00:08:26.750 --> 00:08:28.290 align:start position:0% stop people from synchronizing them well of<00:08:26.930> course<00:08:26.990> we<00:08:27.319> can<00:08:27.349> nuke<00:08:27.680> the<00:08:27.830> boot<00:08:28.009> nodes 00:08:28.290 --> 00:08:28.300 align:start position:0% of course we can nuke the boot nodes 00:08:28.300 --> 00:08:31.089 align:start position:0% of course we can nuke the boot nodes half<00:08:29.300> way<00:08:29.539> offline<00:08:30.020> so<00:08:30.229> that<00:08:30.349> they<00:08:30.469> refuse<00:08:30.979> to 00:08:31.089 --> 00:08:31.099 align:start position:0% half way offline so that they refuse to 00:08:31.099 --> 00:08:32.500 align:start position:0% half way offline so that they refuse to give<00:08:31.250> you<00:08:31.339> the<00:08:31.490> data<00:08:31.669> but<00:08:31.939> that's<00:08:32.120> not<00:08:32.269> nice 00:08:32.500 --> 00:08:32.510 align:start position:0% give you the data but that's not nice 00:08:32.510 --> 00:08:35.350 align:start position:0% give you the data but that's not nice alternatively<00:08:33.440> we<00:08:34.039> need<00:08:34.250> maybe<00:08:35.089> a<00:08:35.180> better 00:08:35.350 --> 00:08:35.360 align:start position:0% alternatively we need maybe a better 00:08:35.360 --> 00:08:37.240 align:start position:0% alternatively we need maybe a better discovery<00:08:35.959> protocol<00:08:36.380> so<00:08:36.529> that<00:08:36.680> you<00:08:36.800> can<00:08:36.949> find 00:08:37.240 --> 00:08:37.250 align:start position:0% discovery protocol so that you can find 00:08:37.250 --> 00:08:41.230 align:start position:0% discovery protocol so that you can find beers<00:08:37.640> faster<00:08:38.209> better<00:08:38.839> peers<00:08:39.260> etc<00:08:39.800> again<00:08:40.430> just 00:08:41.230 --> 00:08:41.240 align:start position:0% beers faster better peers etc again just 00:08:41.240 --> 00:08:43.899 align:start position:0% beers faster better peers etc again just we<00:08:41.360> just<00:08:41.570> looked<00:08:41.750> at<00:08:42.070> what<00:08:43.070> the<00:08:43.519> node<00:08:43.760> is 00:08:43.899 --> 00:08:43.909 align:start position:0% we just looked at what the node is 00:08:43.909 --> 00:08:45.519 align:start position:0% we just looked at what the node is consuming<00:08:44.449> its<00:08:44.570> network<00:08:44.870> bandwidth<00:08:45.230> on<00:08:45.380> and 00:08:45.519 --> 00:08:45.529 align:start position:0% consuming its network bandwidth on and 00:08:45.529 --> 00:08:46.840 align:start position:0% consuming its network bandwidth on and we<00:08:45.620> can<00:08:45.800> immediately<00:08:45.949> make<00:08:46.399> really<00:08:46.699> nice 00:08:46.840 --> 00:08:46.850 align:start position:0% we can immediately make really nice 00:08:46.850 --> 00:08:50.079 align:start position:0% we can immediately make really nice guesses<00:08:47.380> now<00:08:48.380> there<00:08:49.070> are<00:08:49.190> four<00:08:49.430> resource 00:08:50.079 --> 00:08:50.089 align:start position:0% guesses now there are four resource 00:08:50.089 --> 00:08:51.460 align:start position:0% guesses now there are four resource families<00:08:50.570> why<00:08:50.750> we<00:08:50.930> can<00:08:51.110> do<00:08:51.199> a<00:08:51.230> similar 00:08:51.460 --> 00:08:51.470 align:start position:0% families why we can do a similar 00:08:51.470 --> 00:08:54.900 align:start position:0% families why we can do a similar exercise<00:08:51.649> for<00:08:52.430> CPU<00:08:52.820> usage<00:08:53.209> now<00:08:54.019> CPU<00:08:54.440> search<00:08:54.680> if 00:08:54.900 --> 00:08:54.910 align:start position:0% exercise for CPU usage now CPU search if 00:08:54.910 --> 00:08:58.210 align:start position:0% exercise for CPU usage now CPU search if somebody<00:08:55.910> were<00:08:56.269> to<00:08:56.329> ask<00:08:56.569> you<00:08:56.750> what<00:08:57.170> does<00:08:57.350> the 00:08:58.210 --> 00:08:58.220 align:start position:0% somebody were to ask you what does the 00:08:58.220 --> 00:08:59.830 align:start position:0% somebody were to ask you what does the theorem<00:08:58.430> node<00:08:58.670> use<00:08:58.910> CPU<00:08:59.390> for 00:08:59.830 --> 00:08:59.840 align:start position:0% theorem node use CPU for 00:08:59.840 --> 00:09:01.630 align:start position:0% theorem node use CPU for the<00:09:00.260> no-brainer<00:09:00.890> answer<00:09:01.310> is<00:09:01.430> block 00:09:01.630 --> 00:09:01.640 align:start position:0% the no-brainer answer is block 00:09:01.640 --> 00:09:05.080 align:start position:0% the no-brainer answer is block processing<00:09:02.180> and<00:09:02.540> this<00:09:03.440> was<00:09:03.650> our<00:09:04.090> false 00:09:05.080 --> 00:09:05.090 align:start position:0% processing and this was our false 00:09:05.090 --> 00:09:08.530 align:start position:0% processing and this was our false assumption<00:09:05.779> for<00:09:06.170> many<00:09:06.680> years<00:09:07.010> -<00:09:07.250> and<00:09:07.540> about 00:09:08.530 --> 00:09:08.540 align:start position:0% assumption for many years - and about 00:09:08.540 --> 00:09:10.750 align:start position:0% assumption for many years - and about maybe<00:09:09.320> half<00:09:09.560> a<00:09:09.740> year<00:09:09.770> ago<00:09:10.100> somewhere<00:09:10.580> in 00:09:10.750 --> 00:09:10.760 align:start position:0% maybe half a year ago somewhere in 00:09:10.760 --> 00:09:13.360 align:start position:0% maybe half a year ago somewhere in January<00:09:11.210> we<00:09:11.420> decided<00:09:11.930> that<00:09:12.050> okay<00:09:12.650> something's 00:09:13.360 --> 00:09:13.370 align:start position:0% January we decided that okay something's 00:09:13.370 --> 00:09:16.060 align:start position:0% January we decided that okay something's not<00:09:13.550> right<00:09:13.610> so<00:09:14.150> let's<00:09:14.960> try<00:09:15.230> to<00:09:15.290> split<00:09:15.740> that 00:09:16.060 --> 00:09:16.070 align:start position:0% not right so let's try to split that 00:09:16.070 --> 00:09:19.030 align:start position:0% not right so let's try to split that block<00:09:17.029> processing<00:09:17.600> up<00:09:17.810> so<00:09:18.230> we<00:09:18.350> saw<00:09:18.560> that<00:09:18.770> okay 00:09:19.030 --> 00:09:19.040 align:start position:0% block processing up so we saw that okay 00:09:19.040 --> 00:09:20.079 align:start position:0% block processing up so we saw that okay block<00:09:19.250> processing<00:09:19.670> takes<00:09:19.880> a<00:09:20.000> hundred 00:09:20.079 --> 00:09:20.089 align:start position:0% block processing takes a hundred 00:09:20.089 --> 00:09:22.780 align:start position:0% block processing takes a hundred milliseconds<00:09:20.839> but<00:09:20.990> what's<00:09:21.320> inside<00:09:21.560> that<00:09:22.040> and 00:09:22.780 --> 00:09:22.790 align:start position:0% milliseconds but what's inside that and 00:09:22.790 --> 00:09:24.820 align:start position:0% milliseconds but what's inside that and actually<00:09:23.150> figure<00:09:23.330> that<00:09:23.600> well<00:09:23.810> when<00:09:24.560> you<00:09:24.680> run<00:09:24.800> a 00:09:24.820 --> 00:09:24.830 align:start position:0% actually figure that well when you run a 00:09:24.830 --> 00:09:26.950 align:start position:0% actually figure that well when you run a transaction<00:09:25.130> you<00:09:25.760> there<00:09:26.390> is<00:09:26.510> execution 00:09:26.950 --> 00:09:26.960 align:start position:0% transaction you there is execution 00:09:26.960 --> 00:09:29.650 align:start position:0% transaction you there is execution components<00:09:27.680> we<00:09:28.190> also<00:09:28.370> load<00:09:28.730> data<00:09:29.000> from<00:09:29.450> disk 00:09:29.650 --> 00:09:29.660 align:start position:0% components we also load data from disk 00:09:29.660 --> 00:09:31.480 align:start position:0% components we also load data from disk and<00:09:29.870> we<00:09:29.990> also<00:09:30.110> write<00:09:30.410> it<00:09:30.589> to<00:09:30.740> disk<00:09:30.770> so<00:09:31.130> let's 00:09:31.480 --> 00:09:31.490 align:start position:0% and we also write it to disk so let's 00:09:31.490 --> 00:09:33.610 align:start position:0% and we also write it to disk so let's and<00:09:31.670> we<00:09:31.760> also<00:09:31.880> do<00:09:32.120> some<00:09:32.300> hashing<00:09:32.660> so<00:09:33.260> let's<00:09:33.440> try 00:09:33.610 --> 00:09:33.620 align:start position:0% and we also do some hashing so let's try 00:09:33.620 --> 00:09:35.140 align:start position:0% and we also do some hashing so let's try to<00:09:33.680> actually<00:09:33.860> break<00:09:34.220> it<00:09:34.400> up<00:09:34.520> and<00:09:34.700> meter<00:09:34.880> all 00:09:35.140 --> 00:09:35.150 align:start position:0% to actually break it up and meter all 00:09:35.150 --> 00:09:37.570 align:start position:0% to actually break it up and meter all these<00:09:35.330> components<00:09:35.900> individually<00:09:36.170> and<00:09:36.770> then 00:09:37.570 --> 00:09:37.580 align:start position:0% these components individually and then 00:09:37.580 --> 00:09:40.690 align:start position:0% these components individually and then the<00:09:37.730> resulting<00:09:38.150> chart<00:09:38.480> was<00:09:38.750> a<00:09:38.779> bit<00:09:39.110> I<00:09:39.700> don't 00:09:40.690 --> 00:09:40.700 align:start position:0% the resulting chart was a bit I don't 00:09:40.700 --> 00:09:44.140 align:start position:0% the resulting chart was a bit I don't know<00:09:40.940> a<00:09:41.150> bit<00:09:41.420> surprising<00:09:42.339> it<00:09:43.339> turns<00:09:43.640> out<00:09:43.880> that 00:09:44.140 --> 00:09:44.150 align:start position:0% know a bit surprising it turns out that 00:09:44.150 --> 00:09:46.630 align:start position:0% know a bit surprising it turns out that if<00:09:44.360> you<00:09:44.480> run<00:09:44.630> a<00:09:44.660> full<00:09:44.930> node<00:09:45.640> transaction 00:09:46.630 --> 00:09:46.640 align:start position:0% if you run a full node transaction 00:09:46.640 --> 00:09:49.120 align:start position:0% if you run a full node transaction execution<00:09:46.760> I<00:09:47.240> mean<00:09:47.390> actually<00:09:47.930> CPU<00:09:48.680> running 00:09:49.120 --> 00:09:49.130 align:start position:0% execution I mean actually CPU running 00:09:49.130 --> 00:09:52.900 align:start position:0% execution I mean actually CPU running and<00:09:49.400> computing<00:09:50.000> stuff<00:09:50.270> that's<00:09:50.980> 25%<00:09:51.980> max<00:09:52.640> of 00:09:52.900 --> 00:09:52.910 align:start position:0% and computing stuff that's 25% max of 00:09:52.910 --> 00:09:56.560 align:start position:0% and computing stuff that's 25% max of the<00:09:53.900> block<00:09:54.110> processing<00:09:54.650> so<00:09:55.310> 75%<00:09:56.300> of<00:09:56.390> block 00:09:56.560 --> 00:09:56.570 align:start position:0% the block processing so 75% of block 00:09:56.570 --> 00:09:58.690 align:start position:0% the block processing so 75% of block processing<00:09:57.080> is<00:09:57.290> shuffling<00:09:57.830> data<00:09:58.010> around<00:09:58.430> and 00:09:58.690 --> 00:09:58.700 align:start position:0% processing is shuffling data around and 00:09:58.700 --> 00:10:01.030 align:start position:0% processing is shuffling data around and this<00:09:59.630> is<00:09:59.690> an<00:09:59.900> extremely<00:10:00.320> important<00:10:00.770> thing<00:10:00.800> to 00:10:01.030 --> 00:10:01.040 align:start position:0% this is an extremely important thing to 00:10:01.040 --> 00:10:02.800 align:start position:0% this is an extremely important thing to know<00:10:01.190> because<00:10:01.760> all<00:10:02.029> of<00:10:02.060> a<00:10:02.240> sudden<00:10:02.510> you<00:10:02.660> realize 00:10:02.800 --> 00:10:02.810 align:start position:0% know because all of a sudden you realize 00:10:02.810 --> 00:10:05.829 align:start position:0% know because all of a sudden you realize that<00:10:03.080> optimizing<00:10:04.010> the<00:10:04.100> EVM<00:10:04.550> is<00:10:04.790> not<00:10:05.450> that 00:10:05.829 --> 00:10:05.839 align:start position:0% that optimizing the EVM is not that 00:10:05.839 --> 00:10:08.320 align:start position:0% that optimizing the EVM is not that important<00:10:06.470> so<00:10:07.160> optimizing<00:10:07.550> the<00:10:07.880> database 00:10:08.320 --> 00:10:08.330 align:start position:0% important so optimizing the database 00:10:08.330 --> 00:10:11.710 align:start position:0% important so optimizing the database makes<00:10:09.170> a<00:10:09.380> lot<00:10:09.560> more<00:10:09.589> sense<00:10:10.100> than<00:10:10.940> that<00:10:11.570> even 00:10:11.710 --> 00:10:11.720 align:start position:0% makes a lot more sense than that even 00:10:11.720 --> 00:10:13.870 align:start position:0% makes a lot more sense than that even caring<00:10:12.200> about<00:10:12.320> how<00:10:12.650> much<00:10:12.830> one<00:10:13.160> on<00:10:13.339> up<00:10:13.640> code 00:10:13.870 --> 00:10:13.880 align:start position:0% caring about how much one on up code 00:10:13.880 --> 00:10:16.240 align:start position:0% caring about how much one on up code costs<00:10:14.270> or<00:10:14.390> the<00:10:14.420> other<00:10:14.630> and<00:10:14.920> in<00:10:15.920> the<00:10:15.950> case<00:10:16.220> of 00:10:16.240 --> 00:10:16.250 align:start position:0% costs or the other and in the case of 00:10:16.250 --> 00:10:18.090 align:start position:0% costs or the other and in the case of the<00:10:16.460> boot<00:10:16.610> knows<00:10:16.820> this<00:10:17.000> gets<00:10:17.300> even<00:10:17.750> worse 00:10:18.090 --> 00:10:18.100 align:start position:0% the boot knows this gets even worse 00:10:18.100 --> 00:10:20.500 align:start position:0% the boot knows this gets even worse since<00:10:19.100> everybody<00:10:19.490> is<00:10:19.610> constantly<00:10:20.060> hammering 00:10:20.500 --> 00:10:20.510 align:start position:0% since everybody is constantly hammering 00:10:20.510 --> 00:10:22.000 align:start position:0% since everybody is constantly hammering us<00:10:20.630> with<00:10:20.810> requests<00:10:21.320> in<00:10:21.440> the<00:10:21.530> case<00:10:21.680> of<00:10:21.830> boot 00:10:22.000 --> 00:10:22.010 align:start position:0% us with requests in the case of boot 00:10:22.010 --> 00:10:25.630 align:start position:0% us with requests in the case of boot nodes<00:10:22.250> the<00:10:23.089> ileum<00:10:23.570> actual<00:10:23.960> execution<00:10:24.530> is<00:10:24.650> 10% 00:10:25.630 --> 00:10:25.640 align:start position:0% nodes the ileum actual execution is 10% 00:10:25.640 --> 00:10:28.270 align:start position:0% nodes the ileum actual execution is 10% of<00:10:25.880> the<00:10:26.450> block<00:10:26.630> processing<00:10:27.140> time<00:10:27.350> so<00:10:27.830> that's<00:10:28.040> a 00:10:28.270 --> 00:10:28.280 align:start position:0% of the block processing time so that's a 00:10:28.280 --> 00:10:31.690 align:start position:0% of the block processing time so that's a that's<00:10:29.030> kind<00:10:29.390> of<00:10:29.480> scary<00:10:29.650> but<00:10:30.650> again<00:10:31.339> we<00:10:31.520> have<00:10:31.670> a 00:10:31.690 --> 00:10:31.700 align:start position:0% that's kind of scary but again we have a 00:10:31.700 --> 00:10:34.090 align:start position:0% that's kind of scary but again we have a brilliant<00:10:32.230> idea<00:10:33.230> of<00:10:33.320> where<00:10:33.680> we<00:10:33.800> can<00:10:33.980> optimize 00:10:34.090 --> 00:10:34.100 align:start position:0% brilliant idea of where we can optimize 00:10:34.100 --> 00:10:37.120 align:start position:0% brilliant idea of where we can optimize so<00:10:34.640> just<00:10:35.600> a<00:10:35.720> single<00:10:36.050> chart<00:10:36.350> just<00:10:36.589> some 00:10:37.120 --> 00:10:37.130 align:start position:0% so just a single chart just some 00:10:37.130 --> 00:10:39.130 align:start position:0% so just a single chart just some information<00:10:37.760> we<00:10:37.910> can<00:10:38.060> immediately<00:10:38.240> see<00:10:38.720> how 00:10:39.130 --> 00:10:39.140 align:start position:0% information we can immediately see how 00:10:39.140 --> 00:10:42.310 align:start position:0% information we can immediately see how to<00:10:39.200> proceed<00:10:39.650> now<00:10:39.970> other<00:10:40.970> interesting<00:10:41.900> facts 00:10:42.310 --> 00:10:42.320 align:start position:0% to proceed now other interesting facts 00:10:42.320 --> 00:10:44.800 align:start position:0% to proceed now other interesting facts that<00:10:42.350> these<00:10:42.890> charts<00:10:43.370> allowed<00:10:44.000> us<00:10:44.180> to<00:10:44.209> see<00:10:44.510> is 00:10:44.800 --> 00:10:44.810 align:start position:0% that these charts allowed us to see is 00:10:44.810 --> 00:10:47.380 align:start position:0% that these charts allowed us to see is that<00:10:45.790> transaction<00:10:46.790> validation<00:10:47.300> or 00:10:47.380 --> 00:10:47.390 align:start position:0% that transaction validation or 00:10:47.390 --> 00:10:49.630 align:start position:0% that transaction validation or propagation<00:10:47.839> I<00:10:48.020> mean<00:10:48.200> the<00:10:48.560> you<00:10:49.070> know<00:10:49.310> networks 00:10:49.630 --> 00:10:49.640 align:start position:0% propagation I mean the you know networks 00:10:49.640 --> 00:10:50.980 align:start position:0% propagation I mean the you know networks throughput<00:10:50.089> is<00:10:50.240> I<00:10:50.420> don't<00:10:50.480> know<00:10:50.630> something 00:10:50.980 --> 00:10:50.990 align:start position:0% throughput is I don't know something 00:10:50.990 --> 00:10:52.990 align:start position:0% throughput is I don't know something like<00:10:51.110> 20<00:10:51.440> transactions<00:10:52.070> so<00:10:52.310> you<00:10:52.370> would<00:10:52.520> expect 00:10:52.990 --> 00:10:53.000 align:start position:0% like 20 transactions so you would expect 00:10:53.000 --> 00:10:55.480 align:start position:0% like 20 transactions so you would expect that<00:10:53.290> shuffling<00:10:54.290> 20<00:10:54.800> transactions<00:10:55.339> per 00:10:55.480 --> 00:10:55.490 align:start position:0% that shuffling 20 transactions per 00:10:55.490 --> 00:10:58.120 align:start position:0% that shuffling 20 transactions per second<00:10:55.850> around<00:10:55.940> is<00:10:56.240> no<00:10:56.360> big<00:10:56.600> deal<00:10:56.890> except<00:10:57.890> when 00:10:58.120 --> 00:10:58.130 align:start position:0% second around is no big deal except when 00:10:58.130 --> 00:10:59.650 align:start position:0% second around is no big deal except when you<00:10:58.250> chart<00:10:58.520> it<00:10:58.670> out<00:10:58.820> and<00:10:59.089> turns<00:10:59.270> out<00:10:59.450> that<00:10:59.570> the 00:10:59.650 --> 00:10:59.660 align:start position:0% you chart it out and turns out that the 00:10:59.660 --> 00:11:01.449 align:start position:0% you chart it out and turns out that the boot<00:10:59.839> node<00:11:00.020> is<00:11:00.140> receiving<00:11:00.620> 11,000 00:11:01.449 --> 00:11:01.459 align:start position:0% boot node is receiving 11,000 00:11:01.459 --> 00:11:03.730 align:start position:0% boot node is receiving 11,000 transactions<00:11:02.000> per<00:11:02.150> second<00:11:02.600> most<00:11:03.350> of<00:11:03.500> them<00:11:03.620> are 00:11:03.730 --> 00:11:03.740 align:start position:0% transactions per second most of them are 00:11:03.740 --> 00:11:06.460 align:start position:0% transactions per second most of them are duplicates<00:11:04.279> half<00:11:05.209> of<00:11:05.390> them<00:11:05.540> are<00:11:05.660> invalid<00:11:06.200> half 00:11:06.460 --> 00:11:06.470 align:start position:0% duplicates half of them are invalid half 00:11:06.470 --> 00:11:08.440 align:start position:0% duplicates half of them are invalid half of<00:11:06.650> them<00:11:06.830> are<00:11:06.980> or<00:11:07.250> some<00:11:07.670> portion<00:11:07.880> big<00:11:08.120> portion 00:11:08.440 --> 00:11:08.450 align:start position:0% of them are or some portion big portion 00:11:08.450 --> 00:11:10.300 align:start position:0% of them are or some portion big portion of<00:11:08.510> them<00:11:08.720> are<00:11:08.870> underpriced<00:11:09.500> that<00:11:09.890> are<00:11:10.010> get 00:11:10.300 --> 00:11:10.310 align:start position:0% of them are underpriced that are get 00:11:10.310 --> 00:11:13.240 align:start position:0% of them are underpriced that are get rejected<00:11:10.730> but<00:11:11.420> it's<00:11:11.600> really<00:11:12.010> strange<00:11:13.010> and 00:11:13.240 --> 00:11:13.250 align:start position:0% rejected but it's really strange and 00:11:13.250 --> 00:11:14.050 align:start position:0% rejected but it's really strange and when<00:11:13.370> you<00:11:13.520> look<00:11:13.670> at 00:11:14.050 --> 00:11:14.060 align:start position:0% when you look at 00:11:14.060 --> 00:11:15.610 align:start position:0% when you look at that<00:11:14.150> number<00:11:14.510> you<00:11:14.660> all<00:11:14.780> of<00:11:14.870> a<00:11:14.960> sudden<00:11:15.200> realize 00:11:15.610 --> 00:11:15.620 align:start position:0% that number you all of a sudden realize 00:11:15.620 --> 00:11:17.470 align:start position:0% that number you all of a sudden realize that<00:11:15.860> wait<00:11:16.160> so<00:11:16.430> the<00:11:16.550> transaction<00:11:17.060> pool<00:11:17.270> needs 00:11:17.470 --> 00:11:17.480 align:start position:0% that wait so the transaction pool needs 00:11:17.480 --> 00:11:19.330 align:start position:0% that wait so the transaction pool needs to<00:11:17.600> be<00:11:17.720> really<00:11:18.200> optimized<00:11:18.740> for<00:11:19.130> this 00:11:19.330 --> 00:11:19.340 align:start position:0% to be really optimized for this 00:11:19.340 --> 00:11:21.460 align:start position:0% to be really optimized for this throughput<00:11:19.820> so<00:11:20.300> it's<00:11:20.510> not<00:11:20.690> the<00:11:20.990> throughput<00:11:21.290> of 00:11:21.460 --> 00:11:21.470 align:start position:0% throughput so it's not the throughput of 00:11:21.470 --> 00:11:22.990 align:start position:0% throughput so it's not the throughput of the<00:11:21.650> blockchain<00:11:21.890> rather<00:11:22.370> the<00:11:22.550> throughput<00:11:22.970> of 00:11:22.990 --> 00:11:23.000 align:start position:0% the blockchain rather the throughput of 00:11:23.000 --> 00:11:25.270 align:start position:0% the blockchain rather the throughput of the<00:11:23.240> junk<00:11:23.480> data<00:11:23.660> that's<00:11:24.050> coming<00:11:24.230> in<00:11:24.470> that's 00:11:25.270 --> 00:11:25.280 align:start position:0% the junk data that's coming in that's 00:11:25.280 --> 00:11:28.360 align:start position:0% the junk data that's coming in that's defining<00:11:26.240> how<00:11:26.870> much<00:11:27.050> processing<00:11:27.740> we're<00:11:28.190> going 00:11:28.360 --> 00:11:28.370 align:start position:0% defining how much processing we're going 00:11:28.370 --> 00:11:31.000 align:start position:0% defining how much processing we're going to<00:11:28.460> do<00:11:28.810> and<00:11:29.810> of<00:11:30.140> course<00:11:30.200> we<00:11:30.530> have<00:11:30.680> two<00:11:30.860> more 00:11:31.000 --> 00:11:31.010 align:start position:0% to do and of course we have two more 00:11:31.010 --> 00:11:33.610 align:start position:0% to do and of course we have two more other<00:11:31.190> big<00:11:32.030> consumers<00:11:32.600> for<00:11:32.810> the<00:11:32.930> CPU<00:11:33.290> one<00:11:33.530> of 00:11:33.610 --> 00:11:33.620 align:start position:0% other big consumers for the CPU one of 00:11:33.620 --> 00:11:36.550 align:start position:0% other big consumers for the CPU one of them<00:11:33.770> are<00:11:33.860> RPC<00:11:34.340> calls<00:11:34.610> and<00:11:35.170> which<00:11:36.170> we<00:11:36.320> didn't 00:11:36.550 --> 00:11:36.560 align:start position:0% them are RPC calls and which we didn't 00:11:36.560 --> 00:11:38.410 align:start position:0% them are RPC calls and which we didn't chart<00:11:36.800> yet<00:11:37.010> so<00:11:37.520> that's<00:11:37.700> something<00:11:37.970> that<00:11:38.300> we 00:11:38.410 --> 00:11:38.420 align:start position:0% chart yet so that's something that we 00:11:38.420 --> 00:11:40.540 align:start position:0% chart yet so that's something that we want<00:11:38.570> to<00:11:38.630> do<00:11:38.750> in<00:11:38.840> the<00:11:38.930> future<00:11:39.260> and<00:11:39.550> network 00:11:40.540 --> 00:11:40.550 align:start position:0% want to do in the future and network 00:11:40.550 --> 00:11:42.850 align:start position:0% want to do in the future and network handshake<00:11:41.030> requests<00:11:41.600> which<00:11:41.750> might<00:11:42.530> seem<00:11:42.830> like 00:11:42.850 --> 00:11:42.860 align:start position:0% handshake requests which might seem like 00:11:42.860 --> 00:11:45.940 align:start position:0% handshake requests which might seem like why<00:11:43.580> does<00:11:43.970> that<00:11:44.000> even<00:11:44.390> take<00:11:44.600> CPU<00:11:45.050> well<00:11:45.650> if<00:11:45.860> you 00:11:45.940 --> 00:11:45.950 align:start position:0% why does that even take CPU well if you 00:11:45.950 --> 00:11:47.980 align:start position:0% why does that even take CPU well if you run<00:11:46.100> the<00:11:46.280> Raspberry<00:11:46.700> Pi<00:11:46.760> to<00:11:46.940> murder<00:11:47.360> one<00:11:47.630> core 00:11:47.980 --> 00:11:47.990 align:start position:0% run the Raspberry Pi to murder one core 00:11:47.990 --> 00:11:50.050 align:start position:0% run the Raspberry Pi to murder one core from<00:11:48.440> erase<00:11:48.770> verify<00:11:49.160> just<00:11:49.340> doing<00:11:49.910> the 00:11:50.050 --> 00:11:50.060 align:start position:0% from erase verify just doing the 00:11:50.060 --> 00:11:51.640 align:start position:0% from erase verify just doing the cryptographic<00:11:50.360> handshakes<00:11:51.050> in<00:11:51.200> the<00:11:51.290> network 00:11:51.640 --> 00:11:51.650 align:start position:0% cryptographic handshakes in the network 00:11:51.650 --> 00:11:54.190 align:start position:0% cryptographic handshakes in the network so<00:11:51.890> again<00:11:52.790> we<00:11:53.420> need<00:11:53.600> a<00:11:53.660> new<00:11:53.870> discovery 00:11:54.190 --> 00:11:54.200 align:start position:0% so again we need a new discovery 00:11:54.200 --> 00:11:58.330 align:start position:0% so again we need a new discovery protocol<00:11:54.710> to<00:11:54.830> fix<00:11:55.010> it<00:11:55.160> and<00:11:55.720> a<00:11:56.720> big<00:11:57.040> third<00:11:58.040> one 00:11:58.330 --> 00:11:58.340 align:start position:0% protocol to fix it and a big third one 00:11:58.340 --> 00:12:01.450 align:start position:0% protocol to fix it and a big third one third<00:11:59.090> category<00:11:59.390> is<00:11:59.840> disk<00:12:00.170> now<00:12:00.890> in<00:12:01.160> the<00:12:01.280> case 00:12:01.450 --> 00:12:01.460 align:start position:0% third category is disk now in the case 00:12:01.460 --> 00:12:03.280 align:start position:0% third category is disk now in the case of<00:12:01.640> this<00:12:01.790> it<00:12:02.000> is<00:12:02.030> extremely<00:12:02.750> hard<00:12:02.960> to<00:12:03.110> measure 00:12:03.280 --> 00:12:03.290 align:start position:0% of this it is extremely hard to measure 00:12:03.290 --> 00:12:06.550 align:start position:0% of this it is extremely hard to measure because<00:12:04.780> kind<00:12:05.780> of<00:12:05.900> fun<00:12:06.020> you<00:12:06.140> run<00:12:06.290> a<00:12:06.320> program 00:12:06.550 --> 00:12:06.560 align:start position:0% because kind of fun you run a program 00:12:06.560 --> 00:12:08.530 align:start position:0% because kind of fun you run a program you<00:12:06.860> have<00:12:07.010> all<00:12:07.250> these<00:12:07.520> layers<00:12:07.730> on<00:12:08.060> top<00:12:08.090> of<00:12:08.360> one 00:12:08.530 --> 00:12:08.540 align:start position:0% you have all these layers on top of one 00:12:08.540 --> 00:12:10.660 align:start position:0% you have all these layers on top of one another<00:12:08.650> operating<00:12:09.650> systems<00:12:10.010> containers 00:12:10.660 --> 00:12:10.670 align:start position:0% another operating systems containers 00:12:10.670 --> 00:12:13.450 align:start position:0% another operating systems containers libraries<00:12:11.420> and<00:12:11.660> they<00:12:12.200> all<00:12:12.350> like<00:12:12.620> to<00:12:12.920> be<00:12:13.100> smart 00:12:13.450 --> 00:12:13.460 align:start position:0% libraries and they all like to be smart 00:12:13.460 --> 00:12:15.310 align:start position:0% libraries and they all like to be smart and<00:12:13.640> all<00:12:13.730> like<00:12:13.910> to<00:12:14.120> cache<00:12:14.330> and<00:12:14.630> this<00:12:15.050> kind<00:12:15.260> of 00:12:15.310 --> 00:12:15.320 align:start position:0% and all like to cache and this kind of 00:12:15.320 --> 00:12:16.810 align:start position:0% and all like to cache and this kind of means<00:12:15.560> that<00:12:15.830> there's<00:12:16.070> really<00:12:16.430> hard<00:12:16.640> to 00:12:16.810 --> 00:12:16.820 align:start position:0% means that there's really hard to 00:12:16.820 --> 00:12:18.940 align:start position:0% means that there's really hard to measure<00:12:17.000> the<00:12:17.780> thing<00:12:18.020> that<00:12:18.230> you<00:12:18.320> actually<00:12:18.710> want 00:12:18.940 --> 00:12:18.950 align:start position:0% measure the thing that you actually want 00:12:18.950 --> 00:12:20.920 align:start position:0% measure the thing that you actually want to<00:12:19.040> measure<00:12:19.310> we<00:12:20.120> could<00:12:20.270> measure<00:12:20.420> how<00:12:20.540> much 00:12:20.920 --> 00:12:20.930 align:start position:0% to measure we could measure how much 00:12:20.930 --> 00:12:22.810 align:start position:0% to measure we could measure how much data<00:12:21.110> we're<00:12:21.350> pushing<00:12:21.560> the<00:12:21.770> disk<00:12:22.010> but<00:12:22.640> that 00:12:22.810 --> 00:12:22.820 align:start position:0% data we're pushing the disk but that 00:12:22.820 --> 00:12:24.730 align:start position:0% data we're pushing the disk but that depends<00:12:23.240> on<00:12:23.360> how<00:12:23.750> much<00:12:23.900> RAM<00:12:24.200> the<00:12:24.350> operating 00:12:24.730 --> 00:12:24.740 align:start position:0% depends on how much RAM the operating 00:12:24.740 --> 00:12:26.650 align:start position:0% depends on how much RAM the operating system<00:12:25.100> has<00:12:25.220> on<00:12:25.430> how<00:12:25.550> aggressively<00:12:26.150> it<00:12:26.210> caches 00:12:26.650 --> 00:12:26.660 align:start position:0% system has on how aggressively it caches 00:12:26.660 --> 00:12:29.500 align:start position:0% system has on how aggressively it caches we<00:12:27.560> could<00:12:27.740> check<00:12:27.890> how<00:12:28.100> much<00:12:28.250> it<00:12:28.910> could<00:12:29.120> ask<00:12:29.300> the 00:12:29.500 --> 00:12:29.510 align:start position:0% we could check how much it could ask the 00:12:29.510 --> 00:12:31.750 align:start position:0% we could check how much it could ask the operating<00:12:29.870> system<00:12:29.990> to<00:12:30.320> tell<00:12:30.590> it<00:12:30.710> to<00:12:30.860> us<00:12:30.980> but 00:12:31.750 --> 00:12:31.760 align:start position:0% operating system to tell it to us but 00:12:31.760 --> 00:12:33.640 align:start position:0% operating system to tell it to us but that<00:12:31.940> also<00:12:32.120> depends<00:12:32.570> on<00:12:32.810> if<00:12:33.110> you<00:12:33.260> run<00:12:33.410> it<00:12:33.530> in 00:12:33.640 --> 00:12:33.650 align:start position:0% that also depends on if you run it in 00:12:33.650 --> 00:12:35.380 align:start position:0% that also depends on if you run it in darker<00:12:34.040> darker<00:12:34.400> also<00:12:34.670> start<00:12:34.940> screwing<00:12:35.150> around 00:12:35.380 --> 00:12:35.390 align:start position:0% darker darker also start screwing around 00:12:35.390 --> 00:12:39.400 align:start position:0% darker darker also start screwing around with<00:12:35.720> all<00:12:35.990> kinds<00:12:36.320> of<00:12:36.410> caches<00:12:37.250> now<00:12:37.850> if<00:12:38.030> you<00:12:38.410> were 00:12:39.400 --> 00:12:39.410 align:start position:0% with all kinds of caches now if you were 00:12:39.410 --> 00:12:40.690 align:start position:0% with all kinds of caches now if you were after<00:12:39.710> we<00:12:39.800> figured<00:12:40.100> out<00:12:40.250> that<00:12:40.280> we<00:12:40.400> cannot 00:12:40.690 --> 00:12:40.700 align:start position:0% after we figured out that we cannot 00:12:40.700 --> 00:12:43.000 align:start position:0% after we figured out that we cannot reliably<00:12:41.000> measure<00:12:41.390> we<00:12:41.810> decided<00:12:42.440> okay<00:12:42.710> screw 00:12:43.000 --> 00:12:43.010 align:start position:0% reliably measure we decided okay screw 00:12:43.010 --> 00:12:44.590 align:start position:0% reliably measure we decided okay screw it<00:12:43.100> we're<00:12:43.340> going<00:12:43.460> to<00:12:43.580> measure<00:12:43.790> how<00:12:44.180> much<00:12:44.420> data 00:12:44.590 --> 00:12:44.600 align:start position:0% it we're going to measure how much data 00:12:44.600 --> 00:12:47.170 align:start position:0% it we're going to measure how much data we<00:12:44.930> are<00:12:45.170> pushing<00:12:45.440> into<00:12:45.860> the<00:12:45.890> database<00:12:46.400> well<00:12:47.060> it 00:12:47.170 --> 00:12:47.180 align:start position:0% we are pushing into the database well it 00:12:47.180 --> 00:12:48.460 align:start position:0% we are pushing into the database well it turns<00:12:47.360> out<00:12:47.510> that<00:12:47.690> that's<00:12:47.840> completely<00:12:48.350> useless 00:12:48.460 --> 00:12:48.470 align:start position:0% turns out that that's completely useless 00:12:48.470 --> 00:12:50.530 align:start position:0% turns out that that's completely useless too<00:12:48.950> because<00:12:49.550> leveldb<00:12:49.970> has<00:12:50.270> all<00:12:50.420> these 00:12:50.530 --> 00:12:50.540 align:start position:0% too because leveldb has all these 00:12:50.540 --> 00:12:53.110 align:start position:0% too because leveldb has all these background<00:12:50.810> processes<00:12:51.560> around<00:12:52.160> compactions 00:12:53.110 --> 00:12:53.120 align:start position:0% background processes around compactions 00:12:53.120 --> 00:12:55.990 align:start position:0% background processes around compactions and<00:12:53.450> fancy<00:12:54.200> storage<00:12:54.530> models<00:12:54.950> so<00:12:55.460> in<00:12:55.700> the<00:12:55.850> end 00:12:55.990 --> 00:12:56.000 align:start position:0% and fancy storage models so in the end 00:12:56.000 --> 00:12:59.080 align:start position:0% and fancy storage models so in the end we<00:12:56.420> actually<00:12:56.990> needed<00:12:57.590> to<00:12:57.860> patch<00:12:58.160> level<00:12:58.430> DB<00:12:58.880> and 00:12:59.080 --> 00:12:59.090 align:start position:0% we actually needed to patch level DB and 00:12:59.090 --> 00:13:02.080 align:start position:0% we actually needed to patch level DB and ship<00:12:59.420> all<00:12:59.660> our<00:12:59.810> metrics<00:13:00.260> into<00:13:00.470> level<00:13:00.860> DB<00:13:01.160> so 00:13:02.080 --> 00:13:02.090 align:start position:0% ship all our metrics into level DB so 00:13:02.090 --> 00:13:04.270 align:start position:0% ship all our metrics into level DB so that<00:13:02.480> we<00:13:02.600> can<00:13:02.780> get<00:13:02.960> an<00:13:03.080> accurate<00:13:03.470> measurement 00:13:04.270 --> 00:13:04.280 align:start position:0% that we can get an accurate measurement 00:13:04.280 --> 00:13:06.670 align:start position:0% that we can get an accurate measurement and<00:13:05.150> we're<00:13:05.390> actually<00:13:05.540> really<00:13:05.840> happy<00:13:06.260> for<00:13:06.560> the 00:13:06.670 --> 00:13:06.680 align:start position:0% and we're actually really happy for the 00:13:06.680 --> 00:13:08.500 align:start position:0% and we're actually really happy for the author<00:13:06.830> of<00:13:06.980> go<00:13:07.220> level<00:13:07.400> d<00:13:07.640> before<00:13:07.730> allowing<00:13:08.120> him 00:13:08.500 --> 00:13:08.510 align:start position:0% author of go level d before allowing him 00:13:08.510 --> 00:13:10.390 align:start position:0% author of go level d before allowing him allowing<00:13:09.050> us<00:13:09.350> to<00:13:09.380> keep<00:13:09.830> patching<00:13:10.040> his 00:13:10.390 --> 00:13:10.400 align:start position:0% allowing us to keep patching his 00:13:10.400 --> 00:13:15.640 align:start position:0% allowing us to keep patching his database<00:13:12.130> so<00:13:13.210> this<00:13:14.210> one<00:13:14.390> I<00:13:14.420> will<00:13:14.570> probably<00:13:15.200> not 00:13:15.640 --> 00:13:15.650 align:start position:0% database so this one I will probably not 00:13:15.650 --> 00:13:18.460 align:start position:0% database so this one I will probably not go<00:13:15.860> into<00:13:16.220> but<00:13:16.460> the<00:13:16.700> idea<00:13:17.060> is<00:13:17.120> that<00:13:17.630> after<00:13:18.350> you 00:13:18.460 --> 00:13:18.470 align:start position:0% go into but the idea is that after you 00:13:18.470 --> 00:13:21.280 align:start position:0% go into but the idea is that after you found<00:13:18.710> an<00:13:19.250> actual<00:13:19.730> issue<00:13:20.090> so<00:13:20.720> after<00:13:20.960> you<00:13:21.050> found 00:13:21.280 --> 00:13:21.290 align:start position:0% found an actual issue so after you found 00:13:21.290 --> 00:13:23.530 align:start position:0% found an actual issue so after you found that<00:13:21.980> specific<00:13:22.490> metric<00:13:22.820> that<00:13:22.970> is<00:13:23.090> really<00:13:23.390> out 00:13:23.530 --> 00:13:23.540 align:start position:0% that specific metric that is really out 00:13:23.540 --> 00:13:26.740 align:start position:0% that specific metric that is really out of<00:13:23.570> place<00:13:24.410> then<00:13:25.310> you<00:13:25.550> need<00:13:25.700> to<00:13:25.940> find<00:13:26.210> out<00:13:26.480> why 00:13:26.740 --> 00:13:26.750 align:start position:0% of place then you need to find out why 00:13:26.750 --> 00:13:27.850 align:start position:0% of place then you need to find out why it<00:13:26.810> is<00:13:26.990> out<00:13:27.140> of<00:13:27.170> space 00:13:27.850 --> 00:13:27.860 align:start position:0% it is out of space 00:13:27.860 --> 00:13:30.249 align:start position:0% it is out of space of<00:13:28.009> this<00:13:28.339> place<00:13:28.610> here<00:13:28.999> I<00:13:29.029> gave<00:13:29.540> you<00:13:29.660> some<00:13:29.869> ideas 00:13:30.249 --> 00:13:30.259 align:start position:0% of this place here I gave you some ideas 00:13:30.259 --> 00:13:33.009 align:start position:0% of this place here I gave you some ideas so<00:13:30.559> those<00:13:30.740> are<00:13:30.889> good<00:13:31.069> ideas<00:13:31.160> R&R<00:13:32.029> enough<00:13:32.389> but 00:13:33.009 --> 00:13:33.019 align:start position:0% so those are good ideas R&R enough but 00:13:33.019 --> 00:13:34.809 align:start position:0% so those are good ideas R&R enough but if<00:13:33.170> you<00:13:33.259> don't<00:13:33.529> know<00:13:33.769> if<00:13:34.009> you<00:13:34.129> can<00:13:34.339> if<00:13:34.670> you 00:13:34.809 --> 00:13:34.819 align:start position:0% if you don't know if you can if you 00:13:34.819 --> 00:13:36.879 align:start position:0% if you don't know if you can if you don't<00:13:34.939> have<00:13:35.209> any<00:13:35.389> ideas<00:13:35.809> then<00:13:36.529> the<00:13:36.649> only<00:13:36.769> thing 00:13:36.879 --> 00:13:36.889 align:start position:0% don't have any ideas then the only thing 00:13:36.889 --> 00:13:39.160 align:start position:0% don't have any ideas then the only thing you<00:13:37.040> can<00:13:37.369> do<00:13:37.550> is<00:13:37.730> actually<00:13:37.910> to<00:13:38.329> just<00:13:38.569> try<00:13:39.139> to 00:13:39.160 --> 00:13:39.170 align:start position:0% you can do is actually to just try to 00:13:39.170 --> 00:13:41.679 align:start position:0% you can do is actually to just try to look<00:13:39.529> into<00:13:39.829> the<00:13:40.160> details<00:13:40.819> of<00:13:40.999> the<00:13:41.149> algorithms 00:13:41.679 --> 00:13:41.689 align:start position:0% look into the details of the algorithms 00:13:41.689 --> 00:13:43.809 align:start position:0% look into the details of the algorithms internally<00:13:42.319> and<00:13:42.769> try<00:13:42.949> to<00:13:43.009> map<00:13:43.309> out<00:13:43.519> what<00:13:43.670> the 00:13:43.809 --> 00:13:43.819 align:start position:0% internally and try to map out what the 00:13:43.819 --> 00:13:46.689 align:start position:0% internally and try to map out what the algorithms<00:13:44.300> are<00:13:44.449> doing<00:13:44.629> and<00:13:44.990> and<00:13:45.740> try<00:13:46.369> to<00:13:46.429> have 00:13:46.689 --> 00:13:46.699 align:start position:0% algorithms are doing and and try to have 00:13:46.699 --> 00:13:49.059 align:start position:0% algorithms are doing and and try to have better<00:13:47.179> guesses<00:13:47.689> and<00:13:47.839> for<00:13:48.499> example<00:13:48.679> this<00:13:48.920> is 00:13:49.059 --> 00:13:49.069 align:start position:0% better guesses and for example this is 00:13:49.069 --> 00:13:51.369 align:start position:0% better guesses and for example this is maybe<00:13:49.249> one<00:13:49.579> third<00:13:49.910> of<00:13:50.149> the<00:13:50.689> light<00:13:50.839> servers 00:13:51.369 --> 00:13:51.379 align:start position:0% maybe one third of the light servers 00:13:51.379 --> 00:13:54.009 align:start position:0% maybe one third of the light servers charge<00:13:52.220> that<00:13:52.459> Kari<00:13:52.699> put<00:13:52.939> together<00:13:53.119> and<00:13:53.569> there 00:13:54.009 --> 00:13:54.019 align:start position:0% charge that Kari put together and there 00:13:54.019 --> 00:13:55.749 align:start position:0% charge that Kari put together and there are<00:13:54.110> lots<00:13:54.410> and<00:13:54.619> lots<00:13:54.829> of<00:13:54.949> these<00:13:55.129> just<00:13:55.610> to 00:13:55.749 --> 00:13:55.759 align:start position:0% are lots and lots of these just to 00:13:55.759 --> 00:13:58.929 align:start position:0% are lots and lots of these just to figure<00:13:55.910> out<00:13:56.149> individual<00:13:57.079> tiny<00:13:57.319> bugs<00:13:57.730> but<00:13:58.730> I 00:13:58.929 --> 00:13:58.939 align:start position:0% figure out individual tiny bugs but I 00:13:58.939 --> 00:14:00.340 align:start position:0% figure out individual tiny bugs but I won't<00:13:59.209> go<00:13:59.329> into<00:13:59.480> these<00:13:59.660> because<00:13:59.929> that's 00:14:00.340 --> 00:14:00.350 align:start position:0% won't go into these because that's 00:14:00.350 --> 00:14:03.040 align:start position:0% won't go into these because that's completely<00:14:00.800> out<00:14:01.249> of<00:14:01.459> scope<00:14:01.699> here<00:14:02.050> however 00:14:03.040 --> 00:14:03.050 align:start position:0% completely out of scope here however 00:14:03.050 --> 00:14:06.069 align:start position:0% completely out of scope here however let's<00:14:03.769> suppose<00:14:04.040> we<00:14:04.610> did<00:14:04.879> manage<00:14:05.089> to<00:14:05.240> find<00:14:05.809> what 00:14:06.069 --> 00:14:06.079 align:start position:0% let's suppose we did manage to find what 00:14:06.079 --> 00:14:10.119 align:start position:0% let's suppose we did manage to find what all<00:14:06.769> we<00:14:07.160> did<00:14:07.309> manage<00:14:08.170> we<00:14:09.170> have<00:14:09.379> an<00:14:09.619> idea<00:14:10.009> of 00:14:10.119 --> 00:14:10.129 align:start position:0% all we did manage we have an idea of 00:14:10.129 --> 00:14:12.579 align:start position:0% all we did manage we have an idea of what<00:14:10.579> what<00:14:11.509> the<00:14:11.629> anomaly<00:14:11.809> what<00:14:12.170> was<00:14:12.290> the<00:14:12.410> cause 00:14:12.579 --> 00:14:12.589 align:start position:0% what what the anomaly what was the cause 00:14:12.589 --> 00:14:14.470 align:start position:0% what what the anomaly what was the cause of<00:14:12.619> the<00:14:12.860> anomaly<00:14:13.309> and<00:14:13.519> we<00:14:13.910> have<00:14:14.059> a<00:14:14.089> poor 00:14:14.470 --> 00:14:14.480 align:start position:0% of the anomaly and we have a poor 00:14:14.480 --> 00:14:16.900 align:start position:0% of the anomaly and we have a poor request<00:14:14.809> to<00:14:14.929> fix<00:14:15.139> it<00:14:15.319> so<00:14:15.619> the<00:14:16.160> next<00:14:16.550> thing<00:14:16.759> is 00:14:16.900 --> 00:14:16.910 align:start position:0% request to fix it so the next thing is 00:14:16.910 --> 00:14:18.429 align:start position:0% request to fix it so the next thing is we<00:14:17.029> just<00:14:17.059> open<00:14:17.420> a<00:14:17.480> pull<00:14:17.660> request<00:14:17.689> merge<00:14:18.170> it<00:14:18.319> in 00:14:18.429 --> 00:14:18.439 align:start position:0% we just open a pull request merge it in 00:14:18.439 --> 00:14:20.410 align:start position:0% we just open a pull request merge it in and<00:14:18.559> everything<00:14:18.619> is<00:14:18.949> fixed<00:14:19.309> yeah<00:14:19.670> no<00:14:20.029> wrong 00:14:20.410 --> 00:14:20.420 align:start position:0% and everything is fixed yeah no wrong 00:14:20.420 --> 00:14:23.799 align:start position:0% and everything is fixed yeah no wrong the<00:14:21.350> issue<00:14:21.470> is<00:14:21.709> that<00:14:21.980> there<00:14:22.850> are<00:14:22.970> simply<00:14:23.569> so 00:14:23.799 --> 00:14:23.809 align:start position:0% the issue is that there are simply so 00:14:23.809 --> 00:14:25.569 align:start position:0% the issue is that there are simply so many<00:14:24.019> moving<00:14:24.259> components<00:14:24.889> in<00:14:24.949> a<00:14:25.040> theorem<00:14:25.339> note 00:14:25.569 --> 00:14:25.579 align:start position:0% many moving components in a theorem note 00:14:25.579 --> 00:14:28.179 align:start position:0% many moving components in a theorem note that<00:14:25.970> the<00:14:26.240> fact<00:14:26.660> that<00:14:26.869> we<00:14:27.079> fixed<00:14:27.470> one<00:14:27.769> thing 00:14:28.179 --> 00:14:28.189 align:start position:0% that the fact that we fixed one thing 00:14:28.189 --> 00:14:31.269 align:start position:0% that the fact that we fixed one thing might<00:14:28.610> actually<00:14:28.850> break<00:14:29.269> others<00:14:29.800> so<00:14:30.800> the 00:14:31.269 --> 00:14:31.279 align:start position:0% might actually break others so the 00:14:31.279 --> 00:14:32.980 align:start position:0% might actually break others so the lifetime<00:14:31.490> of<00:14:31.699> these<00:14:31.999> performance<00:14:32.660> or<00:14:32.809> these 00:14:32.980 --> 00:14:32.990 align:start position:0% lifetime of these performance or these 00:14:32.990 --> 00:14:35.259 align:start position:0% lifetime of these performance or these anomaly<00:14:33.860> fixing<00:14:34.189> or<00:14:34.370> request<00:14:34.699> kind<00:14:34.939> of<00:14:35.059> look 00:14:35.259 --> 00:14:35.269 align:start position:0% anomaly fixing or request kind of look 00:14:35.269 --> 00:14:37.869 align:start position:0% anomaly fixing or request kind of look like<00:14:35.300> people<00:14:36.290> open<00:14:36.679> a<00:14:36.740> PR<00:14:37.069> and<00:14:37.309> then<00:14:37.730> we 00:14:37.869 --> 00:14:37.879 align:start position:0% like people open a PR and then we 00:14:37.879 --> 00:14:39.910 align:start position:0% like people open a PR and then we actually<00:14:38.269> run<00:14:38.420> a<00:14:38.480> one<00:14:38.779> week<00:14:38.959> long<00:14:39.230> benchmark 00:14:39.910 --> 00:14:39.920 align:start position:0% actually run a one week long benchmark 00:14:39.920 --> 00:14:42.579 align:start position:0% actually run a one week long benchmark to<00:14:40.100> see<00:14:40.129> what<00:14:40.519> does<00:14:40.670> it<00:14:40.819> do<00:14:40.999> now<00:14:41.449> this<00:14:41.899> is<00:14:42.199> the 00:14:42.579 --> 00:14:42.589 align:start position:0% to see what does it do now this is the 00:14:42.589 --> 00:14:44.889 align:start position:0% to see what does it do now this is the perfect<00:14:42.860> case<00:14:43.249> we<00:14:43.550> see<00:14:43.759> something<00:14:44.660> like<00:14:44.779> this 00:14:44.889 --> 00:14:44.899 align:start position:0% perfect case we see something like this 00:14:44.899 --> 00:14:46.360 align:start position:0% perfect case we see something like this maybe<00:14:45.110> once<00:14:45.410> a<00:14:45.589> year<00:14:45.620> these<00:14:45.920> are<00:14:45.980> the<00:14:46.220> miracles 00:14:46.360 --> 00:14:46.370 align:start position:0% maybe once a year these are the miracles 00:14:46.370 --> 00:14:49.809 align:start position:0% maybe once a year these are the miracles of<00:14:47.259> development<00:14:48.259> when<00:14:48.920> this<00:14:49.279> was<00:14:49.459> actually<00:14:49.639> a 00:14:49.809 --> 00:14:49.819 align:start position:0% of development when this was actually a 00:14:49.819 --> 00:14:52.780 align:start position:0% of development when this was actually a pull<00:14:49.999> request<00:14:50.029> created<00:14:50.779> by<00:14:50.899> Garry<00:14:51.790> disabling 00:14:52.780 --> 00:14:52.790 align:start position:0% pull request created by Garry disabling 00:14:52.790 --> 00:14:55.179 align:start position:0% pull request created by Garry disabling some<00:14:53.059> internal<00:14:53.470> data<00:14:54.470> shuffling<00:14:54.949> within 00:14:55.179 --> 00:14:55.189 align:start position:0% some internal data shuffling within 00:14:55.189 --> 00:14:58.179 align:start position:0% some internal data shuffling within level<00:14:55.490> DB<00:14:55.790> and<00:14:56.029> what<00:14:56.660> it<00:14:56.809> here<00:14:57.529> actually<00:14:57.769> the 00:14:58.179 --> 00:14:58.189 align:start position:0% level DB and what it here actually the 00:14:58.189 --> 00:15:00.460 align:start position:0% level DB and what it here actually the green<00:14:58.519> lines<00:14:58.999> were<00:14:59.269> the<00:14:59.480> master<00:14:59.870> and<00:15:00.079> the 00:15:00.460 --> 00:15:00.470 align:start position:0% green lines were the master and the 00:15:00.470 --> 00:15:02.860 align:start position:0% green lines were the master and the yellow<00:15:00.740> lines<00:15:00.980> were<00:15:01.339> Garry's<00:15:02.179> experimental 00:15:02.860 --> 00:15:02.870 align:start position:0% yellow lines were Garry's experimental 00:15:02.870 --> 00:15:06.220 align:start position:0% yellow lines were Garry's experimental PR<00:15:03.429> essentially<00:15:04.429> by<00:15:04.579> just<00:15:04.910> swapping<00:15:05.629> a<00:15:05.990> few 00:15:06.220 --> 00:15:06.230 align:start position:0% PR essentially by just swapping a few 00:15:06.230 --> 00:15:08.350 align:start position:0% PR essentially by just swapping a few variables<00:15:06.649> he<00:15:07.100> actually<00:15:07.429> managed<00:15:07.790> to<00:15:07.999> cut 00:15:08.350 --> 00:15:08.360 align:start position:0% variables he actually managed to cut 00:15:08.360 --> 00:15:10.059 align:start position:0% variables he actually managed to cut down<00:15:08.569> the<00:15:08.929> disk<00:15:09.139> IO<00:15:09.379> by<00:15:09.649> an<00:15:09.769> order<00:15:10.040> of 00:15:10.059 --> 00:15:10.069 align:start position:0% down the disk IO by an order of 00:15:10.069 --> 00:15:13.869 align:start position:0% down the disk IO by an order of magnitude<00:15:10.509> so<00:15:11.509> that<00:15:11.720> was<00:15:11.959> I<00:15:12.439> mean<00:15:13.249> you<00:15:13.670> don't 00:15:13.869 --> 00:15:13.879 align:start position:0% magnitude so that was I mean you don't 00:15:13.879 --> 00:15:16.119 align:start position:0% magnitude so that was I mean you don't get<00:15:14.120> these<00:15:14.449> kinds<00:15:14.839> of<00:15:14.899> charts<00:15:15.199> at<00:15:15.439> all<00:15:15.799> these 00:15:16.119 --> 00:15:16.129 align:start position:0% get these kinds of charts at all these 00:15:16.129 --> 00:15:18.189 align:start position:0% get these kinds of charts at all these are<00:15:16.309> just<00:15:16.490> you<00:15:17.299> don't<00:15:17.480> simply<00:15:17.959> don't<00:15:18.139> believe 00:15:18.189 --> 00:15:18.199 align:start position:0% are just you don't simply don't believe 00:15:18.199 --> 00:15:20.439 align:start position:0% are just you don't simply don't believe them<00:15:18.620> when<00:15:18.649> everything<00:15:19.399> is<00:15:19.519> perfectly<00:15:20.029> better 00:15:20.439 --> 00:15:20.449 align:start position:0% them when everything is perfectly better 00:15:20.449 --> 00:15:24.009 align:start position:0% them when everything is perfectly better than<00:15:20.779> previously<00:15:22.089> what<00:15:23.089> you<00:15:23.240> get<00:15:23.420> a<00:15:23.779> lot<00:15:23.990> more 00:15:24.009 --> 00:15:24.019 align:start position:0% than previously what you get a lot more 00:15:24.019 --> 00:15:26.679 align:start position:0% than previously what you get a lot more often<00:15:24.470> however<00:15:24.709> is<00:15:25.699> these<00:15:25.999> kinds<00:15:26.329> of<00:15:26.420> charts 00:15:26.679 --> 00:15:26.689 align:start position:0% often however is these kinds of charts 00:15:26.689 --> 00:15:28.239 align:start position:0% often however is these kinds of charts this<00:15:26.990> was<00:15:27.199> from<00:15:27.379> a<00:15:27.439> previous<00:15:27.799> monitoring 00:15:28.239 --> 00:15:28.249 align:start position:0% this was from a previous monitoring 00:15:28.249 --> 00:15:30.549 align:start position:0% this was from a previous monitoring system<00:15:28.699> that<00:15:28.850> we<00:15:28.970> had<00:15:29.149> where<00:15:29.899> somebody<00:15:30.259> opened 00:15:30.549 --> 00:15:30.559 align:start position:0% system that we had where somebody opened 00:15:30.559 --> 00:15:32.439 align:start position:0% system that we had where somebody opened the<00:15:30.619> pull<00:15:30.799> request<00:15:30.889> at<00:15:31.279> on<00:15:31.399> Windows<00:15:31.759> Windows 00:15:32.439 --> 00:15:32.449 align:start position:0% the pull request at on Windows Windows 00:15:32.449 --> 00:15:34.480 align:start position:0% the pull request at on Windows Windows really<00:15:32.869> chokes<00:15:33.170> on<00:15:33.439> folders<00:15:33.949> that<00:15:34.129> have<00:15:34.189> many 00:15:34.480 --> 00:15:34.490 align:start position:0% really chokes on folders that have many 00:15:34.490 --> 00:15:37.210 align:start position:0% really chokes on folders that have many files<00:15:34.699> in<00:15:34.939> them<00:15:35.119> so<00:15:35.480> let's<00:15:35.929> change<00:15:36.350> level<00:15:36.920> DB 00:15:37.210 --> 00:15:37.220 align:start position:0% files in them so let's change level DB 00:15:37.220 --> 00:15:39.850 align:start position:0% files in them so let's change level DB to<00:15:37.459> use<00:15:37.610> larger<00:15:38.029> files<00:15:38.269> I<00:15:38.569> mean<00:15:39.439> it<00:15:39.559> seems<00:15:39.769> like 00:15:39.850 --> 00:15:39.860 align:start position:0% to use larger files I mean it seems like 00:15:39.860 --> 00:15:41.450 align:start position:0% to use larger files I mean it seems like a<00:15:40.009> pretty<00:15:40.399> good<00:15:40.790> idea 00:15:41.450 --> 00:15:41.460 align:start position:0% a pretty good idea 00:15:41.460 --> 00:15:43.730 align:start position:0% a pretty good idea why<00:15:42.360> would<00:15:42.540> you<00:15:42.660> use<00:15:42.780> two<00:15:42.960> megabyte<00:15:43.350> files<00:15:43.590> if 00:15:43.730 --> 00:15:43.740 align:start position:0% why would you use two megabyte files if 00:15:43.740 --> 00:15:45.740 align:start position:0% why would you use two megabyte files if you<00:15:43.770> can<00:15:43.980> if<00:15:44.460> you<00:15:44.610> can<00:15:44.640> use<00:15:44.820> 100<00:15:45.390> megabyte 00:15:45.740 --> 00:15:45.750 align:start position:0% you can if you can use 100 megabyte 00:15:45.750 --> 00:15:47.990 align:start position:0% you can if you can use 100 megabyte files<00:15:46.020> so<00:15:46.500> he<00:15:46.650> implemented<00:15:47.100> the<00:15:47.160> PR<00:15:47.490> submitted 00:15:47.990 --> 00:15:48.000 align:start position:0% files so he implemented the PR submitted 00:15:48.000 --> 00:15:50.270 align:start position:0% files so he implemented the PR submitted it<00:15:48.120> and<00:15:48.300> actually<00:15:48.960> also<00:15:49.260> submitted<00:15:49.860> benchmark 00:15:50.270 --> 00:15:50.280 align:start position:0% it and actually also submitted benchmark 00:15:50.280 --> 00:15:52.670 align:start position:0% it and actually also submitted benchmark results<00:15:50.730> that<00:15:50.880> yes<00:15:51.180> the<00:15:51.870> number<00:15:52.140> of<00:15:52.230> files<00:15:52.470> go 00:15:52.670 --> 00:15:52.680 align:start position:0% results that yes the number of files go 00:15:52.680 --> 00:15:54.440 align:start position:0% results that yes the number of files go down<00:15:52.920> the<00:15:53.100> number<00:15:53.340> of<00:15:53.460> use<00:15:53.640> file<00:15:53.880> descriptors 00:15:54.440 --> 00:15:54.450 align:start position:0% down the number of use file descriptors 00:15:54.450 --> 00:15:56.830 align:start position:0% down the number of use file descriptors goes<00:15:54.660> down<00:15:54.930> the<00:15:55.560> PR<00:15:55.860> got<00:15:56.040> two<00:15:56.280> reviews 00:15:56.830 --> 00:15:56.840 align:start position:0% goes down the PR got two reviews 00:15:56.840 --> 00:15:59.150 align:start position:0% goes down the PR got two reviews everybody<00:15:57.840> approved<00:15:58.320> it<00:15:58.470> almost<00:15:58.740> got<00:15:58.950> merged 00:15:59.150 --> 00:15:59.160 align:start position:0% everybody approved it almost got merged 00:15:59.160 --> 00:16:01.670 align:start position:0% everybody approved it almost got merged in<00:15:59.400> and<00:15:59.580> then<00:16:00.270> we<00:16:00.390> said<00:16:00.570> that<00:16:00.690> he<00:16:00.840> I<00:16:00.870> okay<00:16:01.200> do 00:16:01.670 --> 00:16:01.680 align:start position:0% in and then we said that he I okay do 00:16:01.680 --> 00:16:03.050 align:start position:0% in and then we said that he I okay do you<00:16:01.770> know<00:16:01.890> what<00:16:01.980> let's<00:16:02.250> benchmark<00:16:02.850> this 00:16:03.050 --> 00:16:03.060 align:start position:0% you know what let's benchmark this 00:16:03.060 --> 00:16:06.080 align:start position:0% you know what let's benchmark this because<00:16:03.450> it's<00:16:03.860> touching<00:16:04.860> scary<00:16:05.580> stuff 00:16:06.080 --> 00:16:06.090 align:start position:0% because it's touching scary stuff 00:16:06.090 --> 00:16:10.220 align:start position:0% because it's touching scary stuff and<00:16:06.320> then<00:16:07.580> you<00:16:08.580> go<00:16:09.150> you<00:16:09.180> start<00:16:09.690> to<00:16:09.810> look<00:16:09.930> at<00:16:10.110> the 00:16:10.220 --> 00:16:10.230 align:start position:0% and then you go you start to look at the 00:16:10.230 --> 00:16:12.050 align:start position:0% and then you go you start to look at the middle<00:16:10.350> benchmarks<00:16:11.010> and<00:16:11.160> you<00:16:11.310> realize<00:16:11.730> that 00:16:12.050 --> 00:16:12.060 align:start position:0% middle benchmarks and you realize that 00:16:12.060 --> 00:16:14.830 align:start position:0% middle benchmarks and you realize that the<00:16:12.540> disk<00:16:12.780> rights<00:16:13.200> simply<00:16:13.620> exploded<00:16:14.340> so<00:16:14.700> it 00:16:14.830 --> 00:16:14.840 align:start position:0% the disk rights simply exploded so it 00:16:14.840 --> 00:16:19.180 align:start position:0% the disk rights simply exploded so it blew<00:16:15.840> up<00:16:15.990> by<00:16:16.170> two<00:16:16.770> orders<00:16:17.130> of<00:16:17.160> magnitude<00:16:17.310> and 00:16:19.180 --> 00:16:19.190 align:start position:0% blew up by two orders of magnitude and 00:16:19.190 --> 00:16:23.180 align:start position:0% blew up by two orders of magnitude and again<00:16:20.190> it<00:16:20.460> was<00:16:20.970> a<00:16:21.000> perfectly<00:16:21.540> good<00:16:21.810> PR<00:16:22.290> just<00:16:23.040> it 00:16:23.180 --> 00:16:23.190 align:start position:0% again it was a perfectly good PR just it 00:16:23.190 --> 00:16:24.500 align:start position:0% again it was a perfectly good PR just it didn't<00:16:23.400> take<00:16:23.580> into<00:16:23.820> consideration<00:16:24.480> some 00:16:24.500 --> 00:16:24.510 align:start position:0% didn't take into consideration some 00:16:24.510 --> 00:16:26.660 align:start position:0% didn't take into consideration some weird<00:16:25.140> internal<00:16:25.680> thing<00:16:25.890> of<00:16:26.010> leveldb 00:16:26.660 --> 00:16:26.670 align:start position:0% weird internal thing of leveldb 00:16:26.670 --> 00:16:28.640 align:start position:0% weird internal thing of leveldb and<00:16:26.850> these<00:16:27.420> are<00:16:27.570> the<00:16:27.660> PRS<00:16:28.050> that<00:16:28.200> can<00:16:28.350> really 00:16:28.640 --> 00:16:28.650 align:start position:0% and these are the PRS that can really 00:16:28.650 --> 00:16:30.100 align:start position:0% and these are the PRS that can really really<00:16:28.800> bite<00:16:29.100> your<00:16:29.310> performance<00:16:29.910> where 00:16:30.100 --> 00:16:30.110 align:start position:0% really bite your performance where 00:16:30.110 --> 00:16:32.180 align:start position:0% really bite your performance where everything<00:16:31.110> that<00:16:31.260> you<00:16:31.380> do<00:16:31.560> is<00:16:31.740> perfectly 00:16:32.180 --> 00:16:32.190 align:start position:0% everything that you do is perfectly 00:16:32.190 --> 00:16:37.100 align:start position:0% everything that you do is perfectly logical<00:16:32.340> except<00:16:33.150> the<00:16:33.210> result<00:16:34.160> and<00:16:35.810> of<00:16:36.810> course 00:16:37.100 --> 00:16:37.110 align:start position:0% logical except the result and of course 00:16:37.110 --> 00:16:40.040 align:start position:0% logical except the result and of course these<00:16:37.980> are<00:16:38.220> still<00:16:38.580> a<00:16:38.790> good<00:16:39.090> case<00:16:39.360> because<00:16:39.690> this 00:16:40.040 --> 00:16:40.050 align:start position:0% these are still a good case because this 00:16:40.050 --> 00:16:43.490 align:start position:0% these are still a good case because this PR<00:16:40.380> you<00:16:40.500> can<00:16:40.650> at<00:16:40.740> least<00:16:40.770> close<00:16:41.250> and<00:16:42.290> then<00:16:43.290> you 00:16:43.490 --> 00:16:43.500 align:start position:0% PR you can at least close and then you 00:16:43.500 --> 00:16:45.830 align:start position:0% PR you can at least close and then you get<00:16:43.650> the<00:16:43.830> really<00:16:44.010> nasty<00:16:44.220> ones<00:16:44.640> and<00:16:44.910> these<00:16:45.600> are 00:16:45.830 --> 00:16:45.840 align:start position:0% get the really nasty ones and these are 00:16:45.840 --> 00:16:48.470 align:start position:0% get the really nasty ones and these are generally<00:16:46.410> this<00:16:46.560> is<00:16:46.740> the<00:16:46.860> case<00:16:47.220> that<00:16:47.790> we<00:16:48.060> see 00:16:48.470 --> 00:16:48.480 align:start position:0% generally this is the case that we see 00:16:48.480 --> 00:16:52.580 align:start position:0% generally this is the case that we see every<00:16:49.110> time<00:16:50.030> when<00:16:51.030> so<00:16:51.810> the<00:16:52.200> charts<00:16:52.470> that<00:16:52.500> you 00:16:52.580 --> 00:16:52.590 align:start position:0% every time when so the charts that you 00:16:52.590 --> 00:16:55.520 align:start position:0% every time when so the charts that you see<00:16:52.890> here<00:16:53.130> that<00:16:53.760> peak<00:16:54.180> that<00:16:54.600> blue<00:16:54.930> peak<00:16:55.230> that's 00:16:55.520 --> 00:16:55.530 align:start position:0% see here that peak that blue peak that's 00:16:55.530 --> 00:16:57.170 align:start position:0% see here that peak that blue peak that's actually<00:16:55.860> the<00:16:55.950> Shanghai<00:16:56.310> das<00:16:56.520> attack<00:16:56.850> so<00:16:57.060> it<00:16:57.150> I 00:16:57.170 --> 00:16:57.180 align:start position:0% actually the Shanghai das attack so it I 00:16:57.180 --> 00:16:58.730 align:start position:0% actually the Shanghai das attack so it I just<00:16:57.600> closed<00:16:57.870> it<00:16:58.020> on<00:16:58.140> when<00:16:58.530> we<00:16:58.650> were 00:16:58.730 --> 00:16:58.740 align:start position:0% just closed it on when we were 00:16:58.740 --> 00:17:00.140 align:start position:0% just closed it on when we were processing<00:16:59.190> doing<00:16:59.580> a<00:16:59.640> full<00:16:59.850> thing<00:17:00.000> I'm 00:17:00.140 --> 00:17:00.150 align:start position:0% processing doing a full thing I'm 00:17:00.150 --> 00:17:01.460 align:start position:0% processing doing a full thing I'm processing<00:17:00.360> the<00:17:00.690> Shanghai<00:17:01.020> denial<00:17:01.320> of 00:17:01.460 --> 00:17:01.470 align:start position:0% processing the Shanghai denial of 00:17:01.470 --> 00:17:03.080 align:start position:0% processing the Shanghai denial of service<00:17:01.740> attacks<00:17:02.100> and<00:17:02.310> that's<00:17:02.820> the<00:17:02.910> memory 00:17:03.080 --> 00:17:03.090 align:start position:0% service attacks and that's the memory 00:17:03.090 --> 00:17:05.480 align:start position:0% service attacks and that's the memory usage<00:17:03.570> so<00:17:04.320> previously<00:17:04.920> that's<00:17:05.070> the<00:17:05.339> one 00:17:05.480 --> 00:17:05.490 align:start position:0% usage so previously that's the one 00:17:05.490 --> 00:17:07.790 align:start position:0% usage so previously that's the one eighth<00:17:05.700> release<00:17:06.060> family<00:17:06.630> of<00:17:06.720> Gath<00:17:06.959> every<00:17:07.650> time 00:17:07.790 --> 00:17:07.800 align:start position:0% eighth release family of Gath every time 00:17:07.800 --> 00:17:09.440 align:start position:0% eighth release family of Gath every time you<00:17:07.920> reach<00:17:08.100> the<00:17:08.130> Shanghai<00:17:08.850> attacks<00:17:09.150> we<00:17:09.300> had 00:17:09.440 --> 00:17:09.450 align:start position:0% you reach the Shanghai attacks we had 00:17:09.450 --> 00:17:12.290 align:start position:0% you reach the Shanghai attacks we had this<00:17:09.570> huge<00:17:09.810> big<00:17:10.290> huge<00:17:10.740> memory<00:17:10.950> peak<00:17:11.280> simply<00:17:11.670> we 00:17:12.290 --> 00:17:12.300 align:start position:0% this huge big huge memory peak simply we 00:17:12.300 --> 00:17:14.210 align:start position:0% this huge big huge memory peak simply we had<00:17:12.480> various<00:17:12.930> caches<00:17:13.380> in<00:17:13.560> they<00:17:13.740> use<00:17:13.890> a<00:17:13.920> lot<00:17:14.190> of 00:17:14.210 --> 00:17:14.220 align:start position:0% had various caches in they use a lot of 00:17:14.220 --> 00:17:15.980 align:start position:0% had various caches in they use a lot of memory<00:17:14.490> you<00:17:15.240> essentially<00:17:15.630> your<00:17:15.720> machine 00:17:15.980 --> 00:17:15.990 align:start position:0% memory you essentially your machine 00:17:15.990 --> 00:17:18.140 align:start position:0% memory you essentially your machine needed<00:17:16.290> 13<00:17:16.680> gigs<00:17:16.860> of<00:17:17.010> ram<00:17:17.280> to<00:17:17.550> be<00:17:17.910> able<00:17:18.000> to 00:17:18.140 --> 00:17:18.150 align:start position:0% needed 13 gigs of ram to be able to 00:17:18.150 --> 00:17:20.720 align:start position:0% needed 13 gigs of ram to be able to process<00:17:18.420> those<00:17:18.750> blocks<00:17:19.050> and<00:17:19.459> of<00:17:20.459> course 00:17:20.720 --> 00:17:20.730 align:start position:0% process those blocks and of course 00:17:20.730 --> 00:17:22.730 align:start position:0% process those blocks and of course that's<00:17:20.850> a<00:17:21.030> lot<00:17:21.209> so<00:17:21.480> we<00:17:21.839> decided<00:17:22.200> to<00:17:22.350> replace 00:17:22.730 --> 00:17:22.740 align:start position:0% that's a lot so we decided to replace 00:17:22.740 --> 00:17:25.520 align:start position:0% that's a lot so we decided to replace that<00:17:23.430> 13<00:17:23.940> gigs<00:17:24.120> offer<00:17:24.450> I<00:17:24.660> mean<00:17:24.900> that<00:17:25.080> caching 00:17:25.520 --> 00:17:25.530 align:start position:0% that 13 gigs offer I mean that caching 00:17:25.530 --> 00:17:26.660 align:start position:0% that 13 gigs offer I mean that caching algorithm<00:17:26.040> with<00:17:26.160> something<00:17:26.520> completely 00:17:26.660 --> 00:17:26.670 align:start position:0% algorithm with something completely 00:17:26.670 --> 00:17:29.060 align:start position:0% algorithm with something completely different<00:17:27.330> and<00:17:27.510> yeah<00:17:28.320> it<00:17:28.530> was<00:17:28.680> complete 00:17:29.060 --> 00:17:29.070 align:start position:0% different and yeah it was complete 00:17:29.070 --> 00:17:31.490 align:start position:0% different and yeah it was complete success<00:17:29.460> it<00:17:29.640> completely<00:17:30.300> wiped<00:17:30.510> out<00:17:30.810> the<00:17:31.200> the 00:17:31.490 --> 00:17:31.500 align:start position:0% success it completely wiped out the the 00:17:31.500 --> 00:17:34.130 align:start position:0% success it completely wiped out the the peak<00:17:32.100> and<00:17:32.310> also<00:17:33.030> even<00:17:33.330> the<00:17:33.570> general<00:17:33.930> data 00:17:34.130 --> 00:17:34.140 align:start position:0% peak and also even the general data 00:17:34.140 --> 00:17:36.380 align:start position:0% peak and also even the general data usage<00:17:34.650> sorry<00:17:35.130> general<00:17:35.370> memory<00:17:35.760> usage<00:17:36.030> of<00:17:36.180> Gath 00:17:36.380 --> 00:17:36.390 align:start position:0% usage sorry general memory usage of Gath 00:17:36.390 --> 00:17:40.010 align:start position:0% usage sorry general memory usage of Gath went<00:17:36.660> down<00:17:37.340> except<00:17:38.340> the<00:17:38.760> disk<00:17:38.970> usage<00:17:39.210> went<00:17:39.840> up 00:17:40.010 --> 00:17:40.020 align:start position:0% went down except the disk usage went up 00:17:40.020 --> 00:17:42.830 align:start position:0% went down except the disk usage went up by<00:17:40.200> 30%<00:17:40.820> and<00:17:41.820> now<00:17:42.000> you<00:17:42.060> have<00:17:42.390> this<00:17:42.540> huge 00:17:42.830 --> 00:17:42.840 align:start position:0% by 30% and now you have this huge 00:17:42.840 --> 00:17:46.010 align:start position:0% by 30% and now you have this huge problem<00:17:43.260> and<00:17:43.650> question<00:17:44.480> what<00:17:45.480> do<00:17:45.600> you<00:17:45.660> do<00:17:45.840> I 00:17:46.010 --> 00:17:46.020 align:start position:0% problem and question what do you do I 00:17:46.020 --> 00:17:51.110 align:start position:0% problem and question what do you do I mean<00:17:46.530> 30%<00:17:47.520> extra<00:17:47.730> disk<00:17:48.090> i/o<00:17:48.330> is<00:17:48.540> horrible<00:17:50.120> ten 00:17:51.110 --> 00:17:51.120 align:start position:0% mean 30% extra disk i/o is horrible ten 00:17:51.120 --> 00:17:53.090 align:start position:0% mean 30% extra disk i/o is horrible ten extra<00:17:51.540> gigabytes<00:17:52.170> of<00:17:52.290> memory<00:17:52.350> usage<00:17:52.710> is 00:17:53.090 --> 00:17:53.100 align:start position:0% extra gigabytes of memory usage is 00:17:53.100 --> 00:17:54.690 align:start position:0% extra gigabytes of memory usage is equally<00:17:53.460> horrible<00:17:53.790> and 00:17:54.690 --> 00:17:54.700 align:start position:0% equally horrible and 00:17:54.700 --> 00:17:57.240 align:start position:0% equally horrible and you<00:17:54.850> need<00:17:55.720> to<00:17:55.870> make<00:17:56.080> trade-offs<00:17:56.470> you<00:17:56.740> need<00:17:57.160> to 00:17:57.240 --> 00:17:57.250 align:start position:0% you need to make trade-offs you need to 00:17:57.250 --> 00:17:59.190 align:start position:0% you need to make trade-offs you need to make<00:17:57.400> really<00:17:57.670> hard<00:17:58.150> decisions<00:17:58.300> that<00:17:58.870> which<00:17:59.050> is 00:17:59.190 --> 00:17:59.200 align:start position:0% make really hard decisions that which is 00:17:59.200 --> 00:18:01.200 align:start position:0% make really hard decisions that which is better<00:17:59.440> and<00:17:59.740> then<00:18:00.160> in<00:18:00.340> the<00:18:00.430> end<00:18:00.520> we<00:18:00.820> went<00:18:01.000> with 00:18:01.200 --> 00:18:01.210 align:start position:0% better and then in the end we went with 00:18:01.210 --> 00:18:04.320 align:start position:0% better and then in the end we went with the<00:18:02.140> extra<00:18:02.980> memory<00:18:03.130> usage<00:18:03.490> is<00:18:03.610> better<00:18:03.850> simply 00:18:04.320 --> 00:18:04.330 align:start position:0% the extra memory usage is better simply 00:18:04.330 --> 00:18:06.720 align:start position:0% the extra memory usage is better simply because<00:18:04.720> if<00:18:05.620> you<00:18:05.830> use<00:18:05.920> a<00:18:05.950> lot<00:18:06.220> of<00:18:06.340> money<00:18:06.550> if<00:18:06.640> you 00:18:06.720 --> 00:18:06.730 align:start position:0% because if you use a lot of money if you 00:18:06.730 --> 00:18:08.610 align:start position:0% because if you use a lot of money if you want<00:18:07.210> to<00:18:07.300> use<00:18:07.390> more<00:18:07.600> memory<00:18:07.810> than<00:18:07.990> available 00:18:08.610 --> 00:18:08.620 align:start position:0% want to use more memory than available 00:18:08.620 --> 00:18:10.290 align:start position:0% want to use more memory than available you<00:18:08.920> crash<00:18:09.220> I<00:18:09.490> mean<00:18:09.730> it's<00:18:09.850> a<00:18:09.910> heart<00:18:10.120> failure 00:18:10.290 --> 00:18:10.300 align:start position:0% you crash I mean it's a heart failure 00:18:10.300 --> 00:18:12.840 align:start position:0% you crash I mean it's a heart failure whereas<00:18:11.080> if<00:18:11.230> you<00:18:11.380> use<00:18:11.590> more<00:18:12.190> disk<00:18:12.490> than<00:18:12.730> you 00:18:12.840 --> 00:18:12.850 align:start position:0% whereas if you use more disk than you 00:18:12.850 --> 00:18:15.090 align:start position:0% whereas if you use more disk than you would<00:18:13.000> like<00:18:13.210> then<00:18:13.600> things<00:18:13.990> get<00:18:14.170> slower<00:18:14.500> but 00:18:15.090 --> 00:18:15.100 align:start position:0% would like then things get slower but 00:18:15.100 --> 00:18:17.970 align:start position:0% would like then things get slower but they<00:18:15.220> still<00:18:15.460> function<00:18:16.020> so<00:18:17.020> in<00:18:17.170> the<00:18:17.290> end<00:18:17.380> this 00:18:17.970 --> 00:18:17.980 align:start position:0% they still function so in the end this 00:18:17.980 --> 00:18:20.790 align:start position:0% they still function so in the end this is<00:18:18.040> why<00:18:18.310> the<00:18:18.510> actually<00:18:19.510> the<00:18:19.600> won<00:18:19.780> 9-0<00:18:20.200> release 00:18:20.790 --> 00:18:20.800 align:start position:0% is why the actually the won 9-0 release 00:18:20.800 --> 00:18:23.790 align:start position:0% is why the actually the won 9-0 release uses<00:18:21.250> more<00:18:21.610> disk<00:18:22.180> than<00:18:22.630> the<00:18:22.780> won<00:18:22.930> 8<00:18:23.080> whatever 00:18:23.790 --> 00:18:23.800 align:start position:0% uses more disk than the won 8 whatever 00:18:23.800 --> 00:18:26.970 align:start position:0% uses more disk than the won 8 whatever the<00:18:24.160> last<00:18:24.340> one<00:18:24.580> but<00:18:25.270> all<00:18:26.050> in<00:18:26.320> all<00:18:26.410> we<00:18:26.620> think<00:18:26.800> it 00:18:26.970 --> 00:18:26.980 align:start position:0% the last one but all in all we think it 00:18:26.980 --> 00:18:27.750 align:start position:0% the last one but all in all we think it was<00:18:27.190> worth<00:18:27.460> it 00:18:27.750 --> 00:18:27.760 align:start position:0% was worth it 00:18:27.760 --> 00:18:29.280 align:start position:0% was worth it however<00:18:28.030> sins<00:18:28.300> can<00:18:28.450> be<00:18:28.480> fixed<00:18:28.750> it<00:18:28.930> now<00:18:29.050> we<00:18:29.140> were 00:18:29.280 --> 00:18:29.290 align:start position:0% however sins can be fixed it now we were 00:18:29.290 --> 00:18:32.880 align:start position:0% however sins can be fixed it now we were much<00:18:29.590> better<00:18:29.740> so<00:18:31.050> so<00:18:32.050> this<00:18:32.440> thing<00:18:32.590> was<00:18:32.740> fixed 00:18:32.880 --> 00:18:32.890 align:start position:0% much better so so this thing was fixed 00:18:32.890 --> 00:18:37.380 align:start position:0% much better so so this thing was fixed already<00:18:33.510> cool<00:18:34.510> now<00:18:35.050> that<00:18:35.790> is<00:18:36.790> kind<00:18:37.000> of<00:18:37.150> how<00:18:37.330> the 00:18:37.380 --> 00:18:37.390 align:start position:0% already cool now that is kind of how the 00:18:37.390 --> 00:18:41.310 align:start position:0% already cool now that is kind of how the gas<00:18:37.750> team<00:18:37.990> pushes<00:18:38.740> out<00:18:38.920> these<00:18:39.550> are<00:18:40.260> how<00:18:41.260> we 00:18:41.310 --> 00:18:41.320 align:start position:0% gas team pushes out these are how we 00:18:41.320 --> 00:18:43.890 align:start position:0% gas team pushes out these are how we monitor<00:18:41.800> stuff<00:18:42.370> benchmark<00:18:43.330> beers<00:18:43.600> and<00:18:43.750> push 00:18:43.890 --> 00:18:43.900 align:start position:0% monitor stuff benchmark beers and push 00:18:43.900 --> 00:18:46.140 align:start position:0% monitor stuff benchmark beers and push them<00:18:44.080> out<00:18:44.230> now<00:18:44.980> if<00:18:45.250> you<00:18:45.400> would<00:18:45.550> like<00:18:45.700> to<00:18:45.880> repeat 00:18:46.140 --> 00:18:46.150 align:start position:0% them out now if you would like to repeat 00:18:46.150 --> 00:18:47.880 align:start position:0% them out now if you would like to repeat a<00:18:46.210> similar<00:18:46.660> thing<00:18:46.870> in<00:18:47.110> your<00:18:47.290> infrastructure 00:18:47.880 --> 00:18:47.890 align:start position:0% a similar thing in your infrastructure 00:18:47.890 --> 00:18:50.220 align:start position:0% a similar thing in your infrastructure you<00:18:48.640> kind<00:18:48.850> of<00:18:48.940> need<00:18:49.120> to<00:18:49.150> decide<00:18:49.570> how<00:18:49.960> you<00:18:50.020> want 00:18:50.220 --> 00:18:50.230 align:start position:0% you kind of need to decide how you want 00:18:50.230 --> 00:18:52.530 align:start position:0% you kind of need to decide how you want to<00:18:50.320> monitor<00:18:50.530> things<00:18:51.070> you<00:18:51.640> either<00:18:51.670> pay<00:18:52.240> for 00:18:52.530 --> 00:18:52.540 align:start position:0% to monitor things you either pay for 00:18:52.540 --> 00:18:54.270 align:start position:0% to monitor things you either pay for monitoring<00:18:52.930> other<00:18:53.080> service<00:18:53.530> via<00:18:53.740> data<00:18:53.950> that 00:18:54.270 --> 00:18:54.280 align:start position:0% monitoring other service via data that 00:18:54.280 --> 00:18:56.580 align:start position:0% monitoring other service via data that we<00:18:54.460> did<00:18:54.610> that<00:18:54.790> for<00:18:54.970> two<00:18:55.120> years<00:18:55.360> but<00:18:55.690> dated 00:18:56.580 --> 00:18:56.590 align:start position:0% we did that for two years but dated 00:18:56.590 --> 00:18:58.440 align:start position:0% we did that for two years but dated August<00:18:56.860> actually<00:18:56.980> has<00:18:57.280> limitations<00:18:57.700> it 00:18:58.440 --> 00:18:58.450 align:start position:0% August actually has limitations it 00:18:58.450 --> 00:19:00.240 align:start position:0% August actually has limitations it didn't<00:18:58.780> allow<00:18:59.050> us<00:18:59.080> to<00:18:59.260> push<00:18:59.530> all<00:18:59.740> the<00:18:59.950> metrics 00:19:00.240 --> 00:19:00.250 align:start position:0% didn't allow us to push all the metrics 00:19:00.250 --> 00:19:02.370 align:start position:0% didn't allow us to push all the metrics that<00:19:00.340> we<00:19:00.430> had<00:19:00.700> in<00:19:00.940> so<00:19:01.630> in<00:19:01.750> the<00:19:01.870> end<00:19:02.020> we<00:19:02.140> switch 00:19:02.370 --> 00:19:02.380 align:start position:0% that we had in so in the end we switch 00:19:02.380 --> 00:19:04.200 align:start position:0% that we had in so in the end we switch to<00:19:02.530> our<00:19:02.620> own<00:19:02.800> node<00:19:03.100> we<00:19:03.370> just<00:19:03.580> ran<00:19:03.760> graph<00:19:04.000> hana 00:19:04.200 --> 00:19:04.210 align:start position:0% to our own node we just ran graph hana 00:19:04.210 --> 00:19:06.600 align:start position:0% to our own node we just ran graph hana and<00:19:04.480> if<00:19:05.380> you<00:19:05.500> actually<00:19:05.860> decide<00:19:06.130> to<00:19:06.310> run<00:19:06.430> your 00:19:06.600 --> 00:19:06.610 align:start position:0% and if you actually decide to run your 00:19:06.610 --> 00:19:08.910 align:start position:0% and if you actually decide to run your own<00:19:06.640> on-premise<00:19:07.300> instances<00:19:07.930> then<00:19:08.440> you<00:19:08.710> need 00:19:08.910 --> 00:19:08.920 align:start position:0% own on-premise instances then you need 00:19:08.920 --> 00:19:10.830 align:start position:0% own on-premise instances then you need to<00:19:09.040> decide<00:19:09.280> also<00:19:09.670> on<00:19:09.970> a<00:19:10.000> database<00:19:10.390> to<00:19:10.690> push 00:19:10.830 --> 00:19:10.840 align:start position:0% to decide also on a database to push 00:19:10.840 --> 00:19:12.690 align:start position:0% to decide also on a database to push your<00:19:11.020> data<00:19:11.230> in<00:19:11.530> either<00:19:12.130> you<00:19:12.400> go<00:19:12.550> with 00:19:12.690 --> 00:19:12.700 align:start position:0% your data in either you go with 00:19:12.700 --> 00:19:14.910 align:start position:0% your data in either you go with Prometheus<00:19:13.270> or<00:19:13.420> influx<00:19:13.780> DB<00:19:14.080> they<00:19:14.290> say<00:19:14.470> these 00:19:14.910 --> 00:19:14.920 align:start position:0% Prometheus or influx DB they say these 00:19:14.920 --> 00:19:17.340 align:start position:0% Prometheus or influx DB they say these are<00:19:14.980> kind<00:19:15.430> of<00:19:15.550> whatever<00:19:16.000> you<00:19:16.330> prefer<00:19:16.540> the 00:19:17.340 --> 00:19:17.350 align:start position:0% are kind of whatever you prefer the 00:19:17.350 --> 00:19:19.020 align:start position:0% are kind of whatever you prefer the reason<00:19:17.620> I<00:19:17.680> explicitly<00:19:17.980> mention<00:19:18.580> them<00:19:18.730> is<00:19:18.850> that 00:19:19.020 --> 00:19:19.030 align:start position:0% reason I explicitly mention them is that 00:19:19.030 --> 00:19:21.270 align:start position:0% reason I explicitly mention them is that monitoring<00:19:19.780> infrastructure<00:19:20.440> boils<00:19:21.130> down<00:19:21.160> to 00:19:21.270 --> 00:19:21.280 align:start position:0% monitoring infrastructure boils down to 00:19:21.280 --> 00:19:22.920 align:start position:0% monitoring infrastructure boils down to these<00:19:21.550> four<00:19:21.850> things<00:19:22.060> so<00:19:22.360> you<00:19:22.450> don't<00:19:22.720> care 00:19:22.920 --> 00:19:22.930 align:start position:0% these four things so you don't care 00:19:22.930 --> 00:19:24.390 align:start position:0% these four things so you don't care about<00:19:23.050> anything<00:19:23.380> else<00:19:23.650> in<00:19:23.920> the<00:19:24.010> world<00:19:24.040> and 00:19:24.390 --> 00:19:24.400 align:start position:0% about anything else in the world and 00:19:24.400 --> 00:19:27.420 align:start position:0% about anything else in the world and from<00:19:25.390> our<00:19:25.570> part<00:19:25.870> guests<00:19:26.140> can<00:19:26.320> actually<00:19:26.770> push 00:19:27.420 --> 00:19:27.430 align:start position:0% from our part guests can actually push 00:19:27.430 --> 00:19:30.030 align:start position:0% from our part guests can actually push data<00:19:27.790> either<00:19:28.180> it<00:19:28.900> can<00:19:29.200> we<00:19:29.380> can<00:19:29.530> export<00:19:29.860> data 00:19:30.030 --> 00:19:30.040 align:start position:0% data either it can we can export data 00:19:30.040 --> 00:19:31.680 align:start position:0% data either it can we can export data into<00:19:30.190> the<00:19:30.400> data<00:19:30.550> doc<00:19:30.790> format<00:19:31.030> we<00:19:31.390> can<00:19:31.540> push 00:19:31.680 --> 00:19:31.690 align:start position:0% into the data doc format we can push 00:19:31.690 --> 00:19:34.050 align:start position:0% into the data doc format we can push data<00:19:31.870> into<00:19:32.020> gravano<00:19:32.590> and<00:19:32.890> Prometheus<00:19:33.460> too<00:19:33.700> so 00:19:34.050 --> 00:19:34.060 align:start position:0% data into gravano and Prometheus too so 00:19:34.060 --> 00:19:36.150 align:start position:0% data into gravano and Prometheus too so whatever<00:19:34.420> rocks<00:19:34.840> your<00:19:35.050> boat<00:19:35.080> we<00:19:35.710> can<00:19:35.890> do<00:19:36.040> it 00:19:36.150 --> 00:19:36.160 align:start position:0% whatever rocks your boat we can do it 00:19:36.160 --> 00:19:38.700 align:start position:0% whatever rocks your boat we can do it and<00:19:36.460> just<00:19:37.260> just<00:19:38.260> for<00:19:38.440> the<00:19:38.500> sake<00:19:38.680> of 00:19:38.700 --> 00:19:38.710 align:start position:0% and just just for the sake of 00:19:38.710 --> 00:19:41.310 align:start position:0% and just just for the sake of completeness<00:19:39.160> I<00:19:39.550> also<00:19:40.150> linked<00:19:40.480> in<00:19:40.600> our<00:19:40.990> own 00:19:41.310 --> 00:19:41.320 align:start position:0% completeness I also linked in our own 00:19:41.320 --> 00:19:43.830 align:start position:0% completeness I also linked in our own dashboards<00:19:42.070> so<00:19:42.400> we<00:19:42.520> exported<00:19:43.030> them<00:19:43.270> we<00:19:43.720> use 00:19:43.830 --> 00:19:43.840 align:start position:0% dashboards so we exported them we use 00:19:43.840 --> 00:19:45.750 align:start position:0% dashboards so we exported them we use graph<00:19:44.080> on<00:19:44.230> an<00:19:44.320> influx<00:19:44.620> levy<00:19:44.980> but<00:19:45.220> you<00:19:45.640> are 00:19:45.750 --> 00:19:45.760 align:start position:0% graph on an influx levy but you are 00:19:45.760 --> 00:19:48.420 align:start position:0% graph on an influx levy but you are welcome<00:19:46.150> to<00:19:46.300> use<00:19:46.330> them<00:19:46.690> integrate<00:19:47.560> them<00:19:47.770> do 00:19:48.420 --> 00:19:48.430 align:start position:0% welcome to use them integrate them do 00:19:48.430 --> 00:19:49.800 align:start position:0% welcome to use them integrate them do whatever<00:19:48.610> you<00:19:49.030> want<00:19:49.240> with<00:19:49.420> them<00:19:49.510> so<00:19:49.690> all<00:19:49.780> our 00:19:49.800 --> 00:19:49.810 align:start position:0% whatever you want with them so all our 00:19:49.810 --> 00:19:52.050 align:start position:0% whatever you want with them so all our configurations<00:19:50.590> are<00:19:50.800> published<00:19:51.280> there<00:19:51.460> they 00:19:52.050 --> 00:19:52.060 align:start position:0% configurations are published there they 00:19:52.060 --> 00:19:53.790 align:start position:0% configurations are published there they will<00:19:52.180> probably<00:19:52.510> also<00:19:52.630> learn<00:19:52.900> the<00:19:53.080> repo 00:19:53.790 --> 00:19:53.800 align:start position:0% will probably also learn the repo 00:19:53.800 --> 00:19:57.330 align:start position:0% will probably also learn the repo actually<00:19:54.510> cool<00:19:55.510> now<00:19:56.080> what<00:19:56.590> are<00:19:56.830> kind<00:19:57.160> of<00:19:57.280> the 00:19:57.330 --> 00:19:57.340 align:start position:0% actually cool now what are kind of the 00:19:57.340 --> 00:19:59.070 align:start position:0% actually cool now what are kind of the lessons<00:19:57.760> that<00:19:57.880> we<00:19:58.090> learned<00:19:58.330> while<00:19:58.630> creating 00:19:59.070 --> 00:19:59.080 align:start position:0% lessons that we learned while creating 00:19:59.080 --> 00:20:01.680 align:start position:0% lessons that we learned while creating these<00:19:59.350> charts<00:19:59.910> they<00:20:00.910> were<00:20:00.970> that<00:20:01.300> you<00:20:01.480> must 00:20:01.680 --> 00:20:01.690 align:start position:0% these charts they were that you must 00:20:01.690 --> 00:20:03.180 align:start position:0% these charts they were that you must measure<00:20:01.960> as<00:20:02.230> low<00:20:02.440> as<00:20:02.590> you<00:20:02.740> can 00:20:03.180 --> 00:20:03.190 align:start position:0% measure as low as you can 00:20:03.190 --> 00:20:05.340 align:start position:0% measure as low as you can meaning<00:20:04.060> that<00:20:04.090> every<00:20:04.570> abstraction<00:20:05.200> is 00:20:05.340 --> 00:20:05.350 align:start position:0% meaning that every abstraction is 00:20:05.350 --> 00:20:06.990 align:start position:0% meaning that every abstraction is transforms<00:20:05.890> your<00:20:06.010> data<00:20:06.190> everybody<00:20:06.550> tries<00:20:06.820> to 00:20:06.990 --> 00:20:07.000 align:start position:0% transforms your data everybody tries to 00:20:07.000 --> 00:20:08.130 align:start position:0% transforms your data everybody tries to be<00:20:07.120> smart<00:20:07.450> and<00:20:07.630> those 00:20:08.130 --> 00:20:08.140 align:start position:0% be smart and those 00:20:08.140 --> 00:20:10.140 align:start position:0% be smart and those usually<00:20:08.500> ruin<00:20:08.740> your<00:20:08.890> matrix<00:20:09.250> now<00:20:09.820> you<00:20:09.880> always 00:20:10.140 --> 00:20:10.150 align:start position:0% usually ruin your matrix now you always 00:20:10.150 --> 00:20:12.090 align:start position:0% usually ruin your matrix now you always must<00:20:10.300> measure<00:20:10.600> your<00:20:10.960> worst<00:20:11.200> case<00:20:11.440> numbers<00:20:11.890> I 00:20:12.090 --> 00:20:12.100 align:start position:0% must measure your worst case numbers I 00:20:12.100 --> 00:20:14.160 align:start position:0% must measure your worst case numbers I get<00:20:12.700> it<00:20:12.850> that<00:20:13.030> the<00:20:13.180> operating<00:20:13.690> system<00:20:13.810> is 00:20:14.160 --> 00:20:14.170 align:start position:0% get it that the operating system is 00:20:14.170 --> 00:20:15.840 align:start position:0% get it that the operating system is really<00:20:14.320> smart<00:20:14.710> and<00:20:14.950> makes<00:20:15.190> things<00:20:15.580> more 00:20:15.840 --> 00:20:15.850 align:start position:0% really smart and makes things more 00:20:15.850 --> 00:20:18.210 align:start position:0% really smart and makes things more optimal<00:20:16.360> for<00:20:16.570> you<00:20:16.690> but<00:20:17.260> if<00:20:17.530> you<00:20:17.800> assume<00:20:17.890> that 00:20:18.210 --> 00:20:18.220 align:start position:0% optimal for you but if you assume that 00:20:18.220 --> 00:20:19.830 align:start position:0% optimal for you but if you assume that you<00:20:18.310> run<00:20:18.460> out<00:20:18.580> of<00:20:18.610> memory<00:20:18.940> then<00:20:19.420> your<00:20:19.570> worst 00:20:19.830 --> 00:20:19.840 align:start position:0% you run out of memory then your worst 00:20:19.840 --> 00:20:21.690 align:start position:0% you run out of memory then your worst case<00:20:20.050> numbers<00:20:20.440> will<00:20:20.650> do<00:20:20.830> will<00:20:21.250> be<00:20:21.280> the<00:20:21.520> ones 00:20:21.690 --> 00:20:21.700 align:start position:0% case numbers will do will be the ones 00:20:21.700 --> 00:20:23.790 align:start position:0% case numbers will do will be the ones actually<00:20:22.180> hitting<00:20:22.360> you<00:20:22.660> so<00:20:22.690> we<00:20:23.260> always<00:20:23.620> be 00:20:23.790 --> 00:20:23.800 align:start position:0% actually hitting you so we always be 00:20:23.800 --> 00:20:25.410 align:start position:0% actually hitting you so we always be aware<00:20:23.920> of<00:20:24.220> what's<00:20:24.700> the<00:20:24.850> worst<00:20:25.060> that<00:20:25.300> can 00:20:25.410 --> 00:20:25.420 align:start position:0% aware of what's the worst that can 00:20:25.420 --> 00:20:28.020 align:start position:0% aware of what's the worst that can happen<00:20:25.600> and<00:20:26.160> measure<00:20:27.160> everything<00:20:27.550> that<00:20:27.850> can 00:20:28.020 --> 00:20:28.030 align:start position:0% happen and measure everything that can 00:20:28.030 --> 00:20:30.030 align:start position:0% happen and measure everything that can you<00:20:28.390> can<00:20:28.540> afford<00:20:28.840> as<00:20:29.080> I<00:20:29.170> shown<00:20:29.530> you<00:20:29.560> previously 00:20:30.030 --> 00:20:30.040 align:start position:0% you can afford as I shown you previously 00:20:30.040 --> 00:20:32.730 align:start position:0% you can afford as I shown you previously you<00:20:30.970> can<00:20:31.150> gain<00:20:31.420> a<00:20:31.450> lot<00:20:31.780> of<00:20:31.900> information<00:20:32.530> with 00:20:32.730 --> 00:20:32.740 align:start position:0% you can gain a lot of information with 00:20:32.740 --> 00:20:34.350 align:start position:0% you can gain a lot of information with if<00:20:32.980> you<00:20:33.070> have<00:20:33.220> a<00:20:33.250> lots<00:20:33.490> of<00:20:33.640> tiny<00:20:33.880> detailed 00:20:34.350 --> 00:20:34.360 align:start position:0% if you have a lots of tiny detailed 00:20:34.360 --> 00:20:37.560 align:start position:0% if you have a lots of tiny detailed matrix<00:20:34.780> but<00:20:35.610> eventually<00:20:36.610> it<00:20:36.940> will<00:20:37.060> become<00:20:37.390> too 00:20:37.560 --> 00:20:37.570 align:start position:0% matrix but eventually it will become too 00:20:37.570 --> 00:20:39.240 align:start position:0% matrix but eventually it will become too expensive<00:20:37.810> for<00:20:38.380> example<00:20:38.410> Martin<00:20:39.040> did<00:20:39.160> a 00:20:39.240 --> 00:20:39.250 align:start position:0% expensive for example Martin did a 00:20:39.250 --> 00:20:41.880 align:start position:0% expensive for example Martin did a really<00:20:39.400> awesome<00:20:40.530> measurements<00:20:41.530> where<00:20:41.710> he 00:20:41.880 --> 00:20:41.890 align:start position:0% really awesome measurements where he 00:20:41.890 --> 00:20:43.380 align:start position:0% really awesome measurements where he actually<00:20:42.070> measured<00:20:42.610> the<00:20:42.760> cost<00:20:43.030> of<00:20:43.180> each 00:20:43.380 --> 00:20:43.390 align:start position:0% actually measured the cost of each 00:20:43.390 --> 00:20:45.660 align:start position:0% actually measured the cost of each individual<00:20:43.960> opcode<00:20:44.440> it's<00:20:45.040> really<00:20:45.370> insightful 00:20:45.660 --> 00:20:45.670 align:start position:0% individual opcode it's really insightful 00:20:45.670 --> 00:20:47.520 align:start position:0% individual opcode it's really insightful but<00:20:46.150> obviously<00:20:46.690> you<00:20:46.750> cannot<00:20:46.990> run<00:20:47.200> it<00:20:47.290> in<00:20:47.410> live 00:20:47.520 --> 00:20:47.530 align:start position:0% but obviously you cannot run it in live 00:20:47.530 --> 00:20:50.040 align:start position:0% but obviously you cannot run it in live production<00:20:48.010> environment<00:20:48.520> still<00:20:49.180> if<00:20:49.510> you<00:20:49.900> have 00:20:50.040 --> 00:20:50.050 align:start position:0% production environment still if you have 00:20:50.050 --> 00:20:51.600 align:start position:0% production environment still if you have the<00:20:50.170> numbers<00:20:50.500> you<00:20:50.620> can<00:20:50.770> debug<00:20:50.920> an<00:20:51.160> issue<00:20:51.310> if 00:20:51.600 --> 00:20:51.610 align:start position:0% the numbers you can debug an issue if 00:20:51.610 --> 00:20:53.730 align:start position:0% the numbers you can debug an issue if you<00:20:51.760> don't<00:20:51.970> have<00:20:52.120> the<00:20:52.240> numbers<00:20:52.570> you<00:20:53.290> will<00:20:53.500> may 00:20:53.730 --> 00:20:53.740 align:start position:0% you don't have the numbers you will may 00:20:53.740 --> 00:20:55.650 align:start position:0% you don't have the numbers you will may get<00:20:54.220> gather<00:20:54.430> the<00:20:54.670> numbers<00:20:54.970> the<00:20:55.090> next<00:20:55.210> time<00:20:55.480> you 00:20:55.650 --> 00:20:55.660 align:start position:0% get gather the numbers the next time you 00:20:55.660 --> 00:20:57.630 align:start position:0% get gather the numbers the next time you reproduce<00:20:56.110> that<00:20:56.410> issue<00:20:56.560> so<00:20:57.100> there's<00:20:57.340> no<00:20:57.460> way 00:20:57.630 --> 00:20:57.640 align:start position:0% reproduce that issue so there's no way 00:20:57.640 --> 00:20:59.910 align:start position:0% reproduce that issue so there's no way to<00:20:57.700> fix<00:20:58.000> it<00:20:58.180> without<00:20:58.300> numbers<00:20:58.810> yeah<00:20:59.710> and 00:20:59.910 --> 00:20:59.920 align:start position:0% to fix it without numbers yeah and 00:20:59.920 --> 00:21:02.880 align:start position:0% to fix it without numbers yeah and that's<00:21:00.490> about<00:21:00.760> it<00:21:00.910> thank<00:21:01.630> you<00:21:01.690> very<00:21:01.840> much 00:21:02.880 --> 00:21:02.890 align:start position:0% that's about it thank you very much 00:21:02.890 --> 00:21:14.600 align:start position:0% that's about it thank you very much [Applause] 00:21:14.600 --> 00:21:14.610 align:start position:0% 00:21:14.610 --> 00:21:16.670 align:start position:0% you