WEBVTT Kind: captions Language: en 00:00:01.070 --> 00:00:13.120 align:start position:0% [Applause] 00:00:13.120 --> 00:00:13.130 align:start position:0% 00:00:13.130 --> 00:00:17.000 align:start position:0% yeah<00:00:14.540> that's<00:00:15.540> so<00:00:16.200> with<00:00:16.350> that<00:00:16.530> of<00:00:16.680> the<00:00:16.800> way<00:00:16.950> you 00:00:17.000 --> 00:00:17.010 align:start position:0% yeah that's so with that of the way you 00:00:17.010 --> 00:00:21.530 align:start position:0% yeah that's so with that of the way you get<00:00:17.279> started<00:00:18.770> so<00:00:19.770> like<00:00:19.920> I<00:00:20.040> said<00:00:20.279> usually<00:00:21.240> in 00:00:21.530 --> 00:00:21.540 align:start position:0% get started so like I said usually in 00:00:21.540 --> 00:00:25.700 align:start position:0% get started so like I said usually in the<00:00:22.250> information<00:00:23.250> security<00:00:23.460> community<00:00:24.710> the 00:00:25.700 --> 00:00:25.710 align:start position:0% the information security community the 00:00:25.710 --> 00:00:28.609 align:start position:0% the information security community the the<00:00:26.040> focus<00:00:26.310> of<00:00:26.599> security<00:00:27.599> testing<00:00:27.990> is<00:00:28.290> are 00:00:28.609 --> 00:00:28.619 align:start position:0% the focus of security testing is are 00:00:28.619 --> 00:00:30.290 align:start position:0% the focus of security testing is are things<00:00:28.919> like<00:00:29.130> network<00:00:29.640> switches<00:00:30.060> which 00:00:30.290 --> 00:00:30.300 align:start position:0% things like network switches which 00:00:30.300 --> 00:00:33.170 align:start position:0% things like network switches which process<00:00:31.140> external<00:00:32.099> lead<00:00:32.250> and<00:00:32.489> provide<00:00:32.579> access 00:00:33.170 --> 00:00:33.180 align:start position:0% process external lead and provide access 00:00:33.180 --> 00:00:35.960 align:start position:0% process external lead and provide access to<00:00:33.690> prime<00:00:34.079> infrastructure<00:00:34.800> so<00:00:35.370> compiler<00:00:35.760> is 00:00:35.960 --> 00:00:35.970 align:start position:0% to prime infrastructure so compiler is 00:00:35.970 --> 00:00:41.330 align:start position:0% to prime infrastructure so compiler is not<00:00:36.020> let's<00:00:37.020> say<00:00:37.140> a<00:00:37.170> popular<00:00:37.620> target<00:00:38.750> so<00:00:40.220> let<00:00:41.220> me 00:00:41.330 --> 00:00:41.340 align:start position:0% not let's say a popular target so let me 00:00:41.340 --> 00:00:43.580 align:start position:0% not let's say a popular target so let me be<00:00:41.490> clear<00:00:41.520> if<00:00:42.180> that<00:00:42.210> our<00:00:42.570> threat<00:00:42.810> model<00:00:42.840> is<00:00:43.380> not 00:00:43.580 --> 00:00:43.590 align:start position:0% be clear if that our threat model is not 00:00:43.590 --> 00:00:45.890 align:start position:0% be clear if that our threat model is not by<00:00:44.040> the<00:00:44.100> user<00:00:44.400> who's<00:00:44.760> a<00:00:44.790> developer<00:00:45.330> so<00:00:45.780> we 00:00:45.890 --> 00:00:45.900 align:start position:0% by the user who's a developer so we 00:00:45.900 --> 00:00:47.960 align:start position:0% by the user who's a developer so we don't<00:00:46.080> assume<00:00:46.320> that<00:00:46.560> the<00:00:46.770> person<00:00:47.340> using<00:00:47.820> that 00:00:47.960 --> 00:00:47.970 align:start position:0% don't assume that the person using that 00:00:47.970 --> 00:00:50.480 align:start position:0% don't assume that the person using that your<00:00:48.090> father<00:00:48.300> is<00:00:48.480> delicious<00:00:49.020> but<00:00:49.920> rather<00:00:50.190> we 00:00:50.480 --> 00:00:50.490 align:start position:0% your father is delicious but rather we 00:00:50.490 --> 00:00:54.740 align:start position:0% your father is delicious but rather we focus<00:00:50.880> on<00:00:52.430> if<00:00:53.430> the<00:00:53.730> code<00:00:53.970> generated<00:00:54.390> by<00:00:54.690> the 00:00:54.740 --> 00:00:54.750 align:start position:0% focus on if the code generated by the 00:00:54.750 --> 00:00:56.540 align:start position:0% focus on if the code generated by the optimizer<00:00:55.350> is<00:00:55.500> correct<00:00:55.950> right<00:00:56.070> so<00:00:56.370> for 00:00:56.540 --> 00:00:56.550 align:start position:0% optimizer is correct right so for 00:00:56.550 --> 00:00:59.030 align:start position:0% optimizer is correct right so for example<00:00:56.940> if<00:00:57.000> you<00:00:57.150> have<00:00:57.240> a<00:00:57.270> function<00:00:57.780> this<00:00:58.290> is<00:00:58.350> U 00:00:59.030 --> 00:00:59.040 align:start position:0% example if you have a function this is U 00:00:59.040 --> 00:01:01.220 align:start position:0% example if you have a function this is U style<00:00:59.460> program<00:00:59.990> essentially<00:01:00.990> what<00:01:01.110> it's 00:01:01.220 --> 00:01:01.230 align:start position:0% style program essentially what it's 00:01:01.230 --> 00:01:02.750 align:start position:0% style program essentially what it's doing<00:01:01.500> is<00:01:01.620> reminding<00:01:01.860> function<00:01:02.400> foo<00:01:02.580> which 00:01:02.750 --> 00:01:02.760 align:start position:0% doing is reminding function foo which 00:01:02.760 --> 00:01:05.299 align:start position:0% doing is reminding function foo which returns<00:01:03.120> X<00:01:03.480> and<00:01:03.840> essentially<00:01:04.589> setting<00:01:04.799> that<00:01:05.040> X 00:01:05.299 --> 00:01:05.309 align:start position:0% returns X and essentially setting that X 00:01:05.309 --> 00:01:08.740 align:start position:0% returns X and essentially setting that X to<00:01:05.519> me<00:01:05.670> so<00:01:06.659> if<00:01:06.780> you<00:01:06.810> and<00:01:07.320> it's<00:01:07.740> story 00:01:08.740 --> 00:01:08.750 align:start position:0% to me so if you and it's story 00:01:08.750 --> 00:01:12.950 align:start position:0% to me so if you and it's story applications<00:01:09.750> you<00:01:09.960> again<00:01:10.860> in<00:01:11.850> the<00:01:11.970> memory<00:01:12.360> the 00:01:12.950 --> 00:01:12.960 align:start position:0% applications you again in the memory the 00:01:12.960 --> 00:01:15.530 align:start position:0% applications you again in the memory the value<00:01:13.290> returns<00:01:13.650> by<00:01:13.820> so<00:01:14.820> essentially<00:01:15.180> you 00:01:15.530 --> 00:01:15.540 align:start position:0% value returns by so essentially you 00:01:15.540 --> 00:01:18.980 align:start position:0% value returns by so essentially you expect<00:01:15.990> this<00:01:16.140> to<00:01:16.200> store<00:01:17.090> the<00:01:18.090> value<00:01:18.120> 2<00:01:18.630> in<00:01:18.870> the 00:01:18.980 --> 00:01:18.990 align:start position:0% expect this to store the value 2 in the 00:01:18.990 --> 00:01:22.040 align:start position:0% expect this to store the value 2 in the location<00:01:19.380> 0<00:01:19.710> right<00:01:20.370> so<00:01:20.700> and<00:01:21.150> this<00:01:21.630> is<00:01:21.690> what<00:01:21.960> you 00:01:22.040 --> 00:01:22.050 align:start position:0% location 0 right so and this is what you 00:01:22.050 --> 00:01:23.300 align:start position:0% location 0 right so and this is what you can<00:01:22.200> do<00:01:22.290> the<00:01:22.409> right-hand<00:01:22.770> side<00:01:22.950> after 00:01:23.300 --> 00:01:23.310 align:start position:0% can do the right-hand side after 00:01:23.310 --> 00:01:25.700 align:start position:0% can do the right-hand side after optimization<00:01:23.730> which<00:01:24.659> is<00:01:24.870> a<00:01:25.080> suite<00:01:25.320> and<00:01:25.500> store 00:01:25.700 --> 00:01:25.710 align:start position:0% optimization which is a suite and store 00:01:25.710 --> 00:01:27.560 align:start position:0% optimization which is a suite and store co2<00:01:25.950> so<00:01:26.760> essentially<00:01:27.060> if<00:01:27.120> you<00:01:27.180> want<00:01:27.360> to<00:01:27.420> make 00:01:27.560 --> 00:01:27.570 align:start position:0% co2 so essentially if you want to make 00:01:27.570 --> 00:01:31.700 align:start position:0% co2 so essentially if you want to make sure<00:01:27.600> that<00:01:28.250> it's<00:01:29.250> correct<00:01:30.030> code<00:01:30.390> right<00:01:30.630> and<00:01:30.900> it 00:01:31.700 --> 00:01:31.710 align:start position:0% sure that it's correct code right and it 00:01:31.710 --> 00:01:33.320 align:start position:0% sure that it's correct code right and it behaves<00:01:32.010> little<00:01:32.550> pilot<00:01:32.729> behaves<00:01:33.120> to<00:01:33.270> the 00:01:33.320 --> 00:01:33.330 align:start position:0% behaves little pilot behaves to the 00:01:33.330 --> 00:01:36.620 align:start position:0% behaves little pilot behaves to the experiment<00:01:34.790> so<00:01:35.790> for<00:01:36.030> those<00:01:36.120> of<00:01:36.180> you<00:01:36.330> who<00:01:36.570> are 00:01:36.620 --> 00:01:36.630 align:start position:0% experiment so for those of you who are 00:01:36.630 --> 00:01:39.350 align:start position:0% experiment so for those of you who are not<00:01:36.750> familiar<00:01:36.930> with<00:01:37.200> the<00:01:37.409> fast<00:01:37.620> testing<00:01:38.330> for<00:01:39.330> a 00:01:39.350 --> 00:01:39.360 align:start position:0% not familiar with the fast testing for a 00:01:39.360 --> 00:01:42.230 align:start position:0% not familiar with the fast testing for a nutshell<00:01:39.810> is<00:01:40.080> essentially<00:01:40.590> like<00:01:40.770> this<00:01:41.040> you<00:01:41.460> in 00:01:42.230 --> 00:01:42.240 align:start position:0% nutshell is essentially like this you in 00:01:42.240 --> 00:01:44.030 align:start position:0% nutshell is essentially like this you in a<00:01:42.390> in<00:01:42.630> a<00:01:42.690> loop<00:01:42.900> you<00:01:43.200> what<00:01:43.500> you<00:01:43.620> do<00:01:43.770> is<00:01:43.890> you 00:01:44.030 --> 00:01:44.040 align:start position:0% a in a loop you what you do is you 00:01:44.040 --> 00:01:46.160 align:start position:0% a in a loop you what you do is you generate<00:01:44.400> input<00:01:44.850> and<00:01:45.060> fill<00:01:45.450> that 00:01:46.160 --> 00:01:46.170 align:start position:0% generate input and fill that 00:01:46.170 --> 00:01:48.500 align:start position:0% generate input and fill that program<00:01:46.470> under<00:01:46.680> test<00:01:46.920> and<00:01:47.450> essentially<00:01:48.450> you 00:01:48.500 --> 00:01:48.510 align:start position:0% program under test and essentially you 00:01:48.510 --> 00:01:49.850 align:start position:0% program under test and essentially you can<00:01:48.630> do<00:01:48.750> it<00:01:48.870> for<00:01:49.020> as<00:01:49.110> long<00:01:49.170> as<00:01:49.290> you<00:01:49.470> want<00:01:49.710> but 00:01:49.850 --> 00:01:49.860 align:start position:0% can do it for as long as you want but 00:01:49.860 --> 00:01:52.730 align:start position:0% can do it for as long as you want but typically<00:01:50.450> you<00:01:51.450> won't<00:01:51.659> have<00:01:51.840> any<00:01:52.320> games 00:01:52.730 --> 00:01:52.740 align:start position:0% typically you won't have any games 00:01:52.740 --> 00:01:54.800 align:start position:0% typically you won't have any games beyond<00:01:53.159> a<00:01:53.280> certain<00:01:53.430> threshold<00:01:53.580> so<00:01:54.540> at<00:01:54.659> some 00:01:54.800 --> 00:01:54.810 align:start position:0% beyond a certain threshold so at some 00:01:54.810 --> 00:01:56.480 align:start position:0% beyond a certain threshold so at some point<00:01:54.869> you<00:01:55.140> do<00:01:55.290> control<00:01:55.530> C<00:01:55.650> or<00:01:55.860> in<00:01:56.159> the<00:01:56.340> whole 00:01:56.480 --> 00:01:56.490 align:start position:0% point you do control C or in the whole 00:01:56.490 --> 00:02:00.140 align:start position:0% point you do control C or in the whole process<00:01:57.890> it's<00:01:58.890> been<00:01:59.040> shown<00:01:59.909> to<00:02:00.030> be 00:02:00.140 --> 00:02:00.150 align:start position:0% process it's been shown to be 00:02:00.150 --> 00:02:02.380 align:start position:0% process it's been shown to be surprisingly<00:02:00.720> effective<00:02:01.050> at<00:02:01.500> finding<00:02:02.010> bugs 00:02:02.380 --> 00:02:02.390 align:start position:0% surprisingly effective at finding bugs 00:02:02.390 --> 00:02:05.780 align:start position:0% surprisingly effective at finding bugs because<00:02:03.390> most<00:02:03.750> of<00:02:03.930> the<00:02:04.170> new<00:02:04.320> tests<00:02:04.800> capture<00:02:05.400> a 00:02:05.780 --> 00:02:05.790 align:start position:0% because most of the new tests capture a 00:02:05.790 --> 00:02:08.620 align:start position:0% because most of the new tests capture a subset<00:02:06.660> of<00:02:06.690> program<00:02:07.140> behavior<00:02:07.410> and<00:02:07.770> here<00:02:08.130> we 00:02:08.620 --> 00:02:08.630 align:start position:0% subset of program behavior and here we 00:02:08.630 --> 00:02:11.420 align:start position:0% subset of program behavior and here we randomly<00:02:09.630> randomly<00:02:09.990> generate<00:02:10.709> things<00:02:11.100> that 00:02:11.420 --> 00:02:11.430 align:start position:0% randomly randomly generate things that 00:02:11.430 --> 00:02:14.060 align:start position:0% randomly randomly generate things that are<00:02:11.610> usually<00:02:12.120> called<00:02:12.840> cases<00:02:13.290> somehow<00:02:13.650> and 00:02:14.060 --> 00:02:14.070 align:start position:0% are usually called cases somehow and 00:02:14.070 --> 00:02:18.890 align:start position:0% are usually called cases somehow and break<00:02:14.580> the<00:02:14.700> program<00:02:17.239> however<00:02:18.239> applying 00:02:18.890 --> 00:02:18.900 align:start position:0% break the program however applying 00:02:18.900 --> 00:02:22.130 align:start position:0% break the program however applying traditional<00:02:19.590> fuzz<00:02:20.220> testing<00:02:20.400> is<00:02:21.390> limited 00:02:22.130 --> 00:02:22.140 align:start position:0% traditional fuzz testing is limited 00:02:22.140 --> 00:02:24.590 align:start position:0% traditional fuzz testing is limited because<00:02:22.520> our<00:02:23.520> use<00:02:23.790> cases<00:02:24.090> instance 00:02:24.590 --> 00:02:24.600 align:start position:0% because our use cases instance 00:02:24.600 --> 00:02:26.420 align:start position:0% because our use cases instance essentially<00:02:25.260> testing<00:02:25.470> little<00:02:25.830> Python<00:02:26.010> and 00:02:26.420 --> 00:02:26.430 align:start position:0% essentially testing little Python and 00:02:26.430 --> 00:02:28.610 align:start position:0% essentially testing little Python and it's<00:02:27.120> very<00:02:27.360> important<00:02:27.810> to<00:02:27.959> generate<00:02:28.170> valid 00:02:28.610 --> 00:02:28.620 align:start position:0% it's very important to generate valid 00:02:28.620 --> 00:02:30.229 align:start position:0% it's very important to generate valid programs<00:02:29.100> for<00:02:29.280> the<00:02:29.340> compiler<00:02:29.700> which<00:02:30.000> means 00:02:30.229 --> 00:02:30.239 align:start position:0% programs for the compiler which means 00:02:30.239 --> 00:02:32.840 align:start position:0% programs for the compiler which means let's<00:02:30.959> say<00:02:31.080> you<00:02:31.230> have<00:02:31.620> the<00:02:32.100> pilot<00:02:32.400> program<00:02:32.700> on 00:02:32.840 --> 00:02:32.850 align:start position:0% let's say you have the pilot program on 00:02:32.850 --> 00:02:34.370 align:start position:0% let's say you have the pilot program on the<00:02:32.970> left<00:02:33.000> so<00:02:33.360> let<00:02:33.510> me<00:02:33.660> program<00:02:33.989> the<00:02:34.170> left 00:02:34.370 --> 00:02:34.380 align:start position:0% the left so let me program the left 00:02:34.380 --> 00:02:36.350 align:start position:0% the left so let me program the left which<00:02:34.560> defines<00:02:34.950> the<00:02:35.070> contract<00:02:35.489> and<00:02:35.730> a 00:02:36.350 --> 00:02:36.360 align:start position:0% which defines the contract and a 00:02:36.360 --> 00:02:38.060 align:start position:0% which defines the contract and a function<00:02:36.720> to<00:02:36.870> incent<00:02:37.170> the<00:02:37.230> contractor<00:02:37.709> does 00:02:38.060 --> 00:02:38.070 align:start position:0% function to incent the contractor does 00:02:38.070 --> 00:02:41.930 align:start position:0% function to incent the contractor does something<00:02:38.459> right<00:02:39.739> and<00:02:40.739> you<00:02:40.860> apply<00:02:41.130> an<00:02:41.370> effect 00:02:41.930 --> 00:02:41.940 align:start position:0% something right and you apply an effect 00:02:41.940 --> 00:02:43.340 align:start position:0% something right and you apply an effect assume<00:02:42.360> that<00:02:42.540> the<00:02:42.660> process<00:02:42.959> applies<00:02:43.320> a 00:02:43.340 --> 00:02:43.350 align:start position:0% assume that the process applies a 00:02:43.350 --> 00:02:46.160 align:start position:0% assume that the process applies a mutation<00:02:43.850> mutation<00:02:44.850> is<00:02:45.360> essentially<00:02:45.810> any 00:02:46.160 --> 00:02:46.170 align:start position:0% mutation mutation is essentially any 00:02:46.170 --> 00:02:50.150 align:start position:0% mutation mutation is essentially any operation<00:02:46.800> which<00:02:47.150> tweaks<00:02:48.150> bites<00:02:48.870> acts<00:02:49.410> bites 00:02:50.150 --> 00:02:50.160 align:start position:0% operation which tweaks bites acts bites 00:02:50.160 --> 00:02:52.640 align:start position:0% operation which tweaks bites acts bites or<00:02:50.459> mousse<00:02:50.700> pipes<00:02:51.060> from<00:02:51.329> the<00:02:51.390> bike<00:02:51.660> screen<00:02:51.989> on 00:02:52.640 --> 00:02:52.650 align:start position:0% or mousse pipes from the bike screen on 00:02:52.650 --> 00:02:54.890 align:start position:0% or mousse pipes from the bike screen on the<00:02:52.799> left<00:02:53.010> so<00:02:53.220> you<00:02:53.549> can<00:02:53.730> think<00:02:53.880> so<00:02:54.540> the<00:02:54.660> father 00:02:54.890 --> 00:02:54.900 align:start position:0% the left so you can think so the father 00:02:54.900 --> 00:02:57.680 align:start position:0% the left so you can think so the father sees<00:02:55.230> the<00:02:56.040> input<00:02:56.430> as<00:02:56.610> a<00:02:56.640> stream<00:02:57.000> of<00:02:57.120> bytes<00:02:57.299> or 00:02:57.680 --> 00:02:57.690 align:start position:0% sees the input as a stream of bytes or 00:02:57.690 --> 00:03:01.460 align:start position:0% sees the input as a stream of bytes or on<00:02:58.620> the<00:02:58.739> left<00:02:58.920> and<00:02:59.130> it's<00:02:59.489> two<00:02:59.670> weeks<00:03:00.290> by<00:03:01.290> its 00:03:01.460 --> 00:03:01.470 align:start position:0% on the left and it's two weeks by its 00:03:01.470 --> 00:03:03.680 align:start position:0% on the left and it's two weeks by its and<00:03:01.650> Hayek's<00:03:01.920> removes<00:03:02.370> so<00:03:02.850> you<00:03:02.940> can<00:03:03.329> create<00:03:03.540> a 00:03:03.680 --> 00:03:03.690 align:start position:0% and Hayek's removes so you can create a 00:03:03.690 --> 00:03:06.140 align:start position:0% and Hayek's removes so you can create a mutation<00:03:04.049> like<00:03:04.410> the<00:03:05.310> code<00:03:05.760> shown<00:03:05.940> on<00:03:06.030> the 00:03:06.140 --> 00:03:06.150 align:start position:0% mutation like the code shown on the 00:03:06.150 --> 00:03:09.500 align:start position:0% mutation like the code shown on the right<00:03:06.329> which<00:03:07.230> basically<00:03:08.220> tweaks<00:03:08.700> the 00:03:09.500 --> 00:03:09.510 align:start position:0% right which basically tweaks the 00:03:09.510 --> 00:03:11.800 align:start position:0% right which basically tweaks the keyboard<00:03:09.870> function<00:03:10.290> and<00:03:10.530> public<00:03:11.100> right 00:03:11.800 --> 00:03:11.810 align:start position:0% keyboard function and public right 00:03:11.810 --> 00:03:14.449 align:start position:0% keyboard function and public right because<00:03:12.810> it's<00:03:12.930> documented<00:03:13.650> so<00:03:14.250> you<00:03:14.310> can 00:03:14.449 --> 00:03:14.459 align:start position:0% because it's documented so you can 00:03:14.459 --> 00:03:18.180 align:start position:0% because it's documented so you can imagine<00:03:15.079> with<00:03:16.079> a<00:03:16.110> very<00:03:16.410> high<00:03:16.650> likelihood 00:03:18.180 --> 00:03:18.190 align:start position:0% imagine with a very high likelihood 00:03:18.190 --> 00:03:23.160 align:start position:0% imagine with a very high likelihood at<00:03:18.730> the<00:03:19.720> at<00:03:20.620> the<00:03:21.370> input<00:03:21.670> levels<00:03:22.060> is<00:03:22.780> going<00:03:23.110> to 00:03:23.160 --> 00:03:23.170 align:start position:0% at the at the input levels is going to 00:03:23.170 --> 00:03:25.410 align:start position:0% at the at the input levels is going to be<00:03:23.260> simply<00:03:23.500> rejected<00:03:24.010> by<00:03:24.160> the<00:03:24.220> pasa<00:03:24.490> and<00:03:24.970> we 00:03:25.410 --> 00:03:25.420 align:start position:0% be simply rejected by the pasa and we 00:03:25.420 --> 00:03:28.860 align:start position:0% be simply rejected by the pasa and we don't<00:03:25.600> want<00:03:25.840> that<00:03:25.960> because<00:03:26.140> we<00:03:26.530> know<00:03:27.870> the 00:03:28.860 --> 00:03:28.870 align:start position:0% don't want that because we know the 00:03:28.870 --> 00:03:30.870 align:start position:0% don't want that because we know the program<00:03:29.230> to<00:03:29.350> be<00:03:29.380> parse<00:03:29.770> and<00:03:30.040> then<00:03:30.190> optimized 00:03:30.870 --> 00:03:30.880 align:start position:0% program to be parse and then optimized 00:03:30.880 --> 00:03:34.200 align:start position:0% program to be parse and then optimized and<00:03:31.090> then<00:03:31.330> test<00:03:31.540> the<00:03:31.690> optimizer<00:03:32.430> so<00:03:33.430> this 00:03:34.200 --> 00:03:34.210 align:start position:0% and then test the optimizer so this 00:03:34.210 --> 00:03:36.630 align:start position:0% and then test the optimizer so this clearly<00:03:34.540> won't<00:03:34.690> work<00:03:34.960> or<00:03:35.230> it<00:03:35.770> works<00:03:36.100> but<00:03:36.460> it's 00:03:36.630 --> 00:03:36.640 align:start position:0% clearly won't work or it works but it's 00:03:36.640 --> 00:03:42.330 align:start position:0% clearly won't work or it works but it's not<00:03:36.850> very<00:03:37.060> efficient<00:03:39.330> so<00:03:40.710> basically<00:03:41.710> the 00:03:42.330 --> 00:03:42.340 align:start position:0% not very efficient so basically the 00:03:42.340 --> 00:03:45.120 align:start position:0% not very efficient so basically the learning<00:03:42.730> is<00:03:42.910> that<00:03:43.740> nothing<00:03:44.740> will<00:03:44.800> compile 00:03:45.120 --> 00:03:45.130 align:start position:0% learning is that nothing will compile 00:03:45.130 --> 00:03:46.860 align:start position:0% learning is that nothing will compile the<00:03:45.220> required<00:03:45.610> standard<00:03:46.120> invalid<00:03:46.540> programs 00:03:46.860 --> 00:03:46.870 align:start position:0% the required standard invalid programs 00:03:46.870 --> 00:03:48.930 align:start position:0% the required standard invalid programs and<00:03:47.200> now<00:03:47.620> generating<00:03:48.220> valid<00:03:48.460> programs 00:03:48.930 --> 00:03:48.940 align:start position:0% and now generating valid programs 00:03:48.940 --> 00:03:51.270 align:start position:0% and now generating valid programs requires<00:03:49.540> some<00:03:50.530> sort<00:03:50.800> of<00:03:50.860> structure 00:03:51.270 --> 00:03:51.280 align:start position:0% requires some sort of structure 00:03:51.280 --> 00:03:55.680 align:start position:0% requires some sort of structure awareness<00:03:53.130> so<00:03:54.130> let<00:03:54.850> me<00:03:54.970> talk<00:03:55.120> about<00:03:55.180> more 00:03:55.680 --> 00:03:55.690 align:start position:0% awareness so let me talk about more 00:03:55.690 --> 00:03:57.870 align:start position:0% awareness so let me talk about more structure<00:03:56.140> awareness<00:03:56.590> and<00:03:56.800> how<00:03:57.280> we<00:03:57.550> approach 00:03:57.870 --> 00:03:57.880 align:start position:0% structure awareness and how we approach 00:03:57.880 --> 00:04:01.290 align:start position:0% structure awareness and how we approach this<00:03:58.060> problem<00:03:59.610> so<00:04:00.610> essentially<00:04:00.940> we<00:04:01.060> start 00:04:01.290 --> 00:04:01.300 align:start position:0% this problem so essentially we start 00:04:01.300 --> 00:04:05.040 align:start position:0% this problem so essentially we start with<00:04:01.480> a<00:04:01.930> high<00:04:02.020> level<00:04:02.790> specification<00:04:04.050> the 00:04:05.040 --> 00:04:05.050 align:start position:0% with a high level specification the 00:04:05.050 --> 00:04:06.960 align:start position:0% with a high level specification the specification<00:04:05.740> is<00:04:05.920> written<00:04:06.340> in<00:04:06.700> the 00:04:06.960 --> 00:04:06.970 align:start position:0% specification is written in the 00:04:06.970 --> 00:04:09.180 align:start position:0% specification is written in the interface<00:04:07.900> description<00:04:08.200> language<00:04:08.590> called 00:04:09.180 --> 00:04:09.190 align:start position:0% interface description language called 00:04:09.190 --> 00:04:12.150 align:start position:0% interface description language called protocol<00:04:09.520> of<00:04:10.300> protocol<00:04:10.660> buffers<00:04:11.290> this<00:04:11.980> is<00:04:12.070> a 00:04:12.150 --> 00:04:12.160 align:start position:0% protocol of protocol buffers this is a 00:04:12.160 --> 00:04:14.940 align:start position:0% protocol of protocol buffers this is a movie<00:04:12.630> it<00:04:13.630> was<00:04:13.660> personally<00:04:14.410> developed<00:04:14.800> by 00:04:14.940 --> 00:04:14.950 align:start position:0% movie it was personally developed by 00:04:14.950 --> 00:04:17.340 align:start position:0% movie it was personally developed by Google<00:04:15.400> but<00:04:15.820> it's<00:04:16.120> used<00:04:16.660> for<00:04:16.810> various 00:04:17.340 --> 00:04:17.350 align:start position:0% Google but it's used for various 00:04:17.350 --> 00:04:21.500 align:start position:0% Google but it's used for various purposes<00:04:17.940> also<00:04:18.940> in<00:04:19.180> guessing<00:04:19.510> so<00:04:20.280> essentially 00:04:21.500 --> 00:04:21.510 align:start position:0% purposes also in guessing so essentially 00:04:21.510 --> 00:04:25.940 align:start position:0% purposes also in guessing so essentially in<00:04:22.510> the<00:04:22.660> form<00:04:22.870> of<00:04:23.020> language<00:04:23.260> you<00:04:23.680> can<00:04:24.120> define 00:04:25.940 --> 00:04:25.950 align:start position:0% in the form of language you can define 00:04:25.950 --> 00:04:30.240 align:start position:0% in the form of language you can define units<00:04:26.950> of<00:04:27.100> data<00:04:27.280> as<00:04:27.750> messages<00:04:29.070> and<00:04:30.070> each 00:04:30.240 --> 00:04:30.250 align:start position:0% units of data as messages and each 00:04:30.250 --> 00:04:35.460 align:start position:0% units of data as messages and each message<00:04:30.700> contains<00:04:31.470> one<00:04:32.470> or<00:04:32.500> more<00:04:34.169> fields<00:04:35.169> from 00:04:35.460 --> 00:04:35.470 align:start position:0% message contains one or more fields from 00:04:35.470 --> 00:04:39.420 align:start position:0% message contains one or more fields from other<00:04:35.770> messages<00:04:36.250> so<00:04:36.880> for<00:04:37.060> example<00:04:38.430> it's 00:04:39.420 --> 00:04:39.430 align:start position:0% other messages so for example it's 00:04:39.430 --> 00:04:41.940 align:start position:0% other messages so for example it's useful<00:04:39.940> to<00:04:40.060> talk<00:04:40.240> of<00:04:40.510> the<00:04:40.930> specification<00:04:41.710> of 00:04:41.940 --> 00:04:41.950 align:start position:0% useful to talk of the specification of 00:04:41.950 --> 00:04:43.770 align:start position:0% useful to talk of the specification of the<00:04:42.010> you<00:04:42.490> programming<00:04:43.000> language<00:04:43.300> that<00:04:43.330> we 00:04:43.770 --> 00:04:43.780 align:start position:0% the you programming language that we 00:04:43.780 --> 00:04:46.680 align:start position:0% the you programming language that we tested<00:04:44.169> in<00:04:44.440> a<00:04:44.950> top-down<00:04:45.280> fashion<00:04:45.550> at<00:04:45.970> the<00:04:46.480> very 00:04:46.680 --> 00:04:46.690 align:start position:0% tested in a top-down fashion at the very 00:04:46.690 --> 00:04:48.900 align:start position:0% tested in a top-down fashion at the very top<00:04:46.930> you<00:04:47.140> have<00:04:47.380> a<00:04:48.100> program 00:04:48.900 --> 00:04:48.910 align:start position:0% top you have a program 00:04:48.910 --> 00:04:51.570 align:start position:0% top you have a program so<00:04:49.120> the<00:04:49.330> message<00:04:49.630> program<00:04:50.020> which<00:04:50.710> contains<00:04:51.250> a 00:04:51.570 --> 00:04:51.580 align:start position:0% so the message program which contains a 00:04:51.580 --> 00:04:53.880 align:start position:0% so the message program which contains a repeated<00:04:52.180> sequence<00:04:52.660> of<00:04:52.990> a<00:04:53.230> message<00:04:53.590> for<00:04:53.830> a 00:04:53.880 --> 00:04:53.890 align:start position:0% repeated sequence of a message for a 00:04:53.890 --> 00:04:57.690 align:start position:0% repeated sequence of a message for a block<00:04:54.690> which<00:04:55.690> we<00:04:55.900> also<00:04:56.140> defined<00:04:56.590> as<00:04:57.160> a 00:04:57.690 --> 00:04:57.700 align:start position:0% block which we also defined as a 00:04:57.700 --> 00:05:01.530 align:start position:0% block which we also defined as a sequence<00:04:58.360> of<00:04:58.780> a<00:04:59.770> message<00:05:00.490> called<00:05:00.790> statement 00:05:01.530 --> 00:05:01.540 align:start position:0% sequence of a message called statement 00:05:01.540 --> 00:05:04.800 align:start position:0% sequence of a message called statement and<00:05:02.040> so<00:05:03.040> on<00:05:03.190> and<00:05:03.340> so<00:05:03.460> forth<00:05:03.700> so<00:05:03.910> you<00:05:04.030> can<00:05:04.120> you 00:05:04.800 --> 00:05:04.810 align:start position:0% and so on and so forth so you can you 00:05:04.810 --> 00:05:07.860 align:start position:0% and so on and so forth so you can you could<00:05:05.050> although<00:05:05.980> it's<00:05:06.100> not<00:05:06.280> sure 00:05:07.860 --> 00:05:07.870 align:start position:0% could although it's not sure 00:05:07.870 --> 00:05:11.700 align:start position:0% could although it's not sure slightly<00:05:08.290> you<00:05:08.650> could<00:05:10.139> define<00:05:11.139> a<00:05:11.199> message 00:05:11.700 --> 00:05:11.710 align:start position:0% slightly you could define a message 00:05:11.710 --> 00:05:15.060 align:start position:0% slightly you could define a message called<00:05:12.040> its<00:05:12.370> statement<00:05:12.970> a<00:05:13.840> for<00:05:14.350> statement<00:05:14.470> and 00:05:15.060 --> 00:05:15.070 align:start position:0% called its statement a for statement and 00:05:15.070 --> 00:05:16.980 align:start position:0% called its statement a for statement and so<00:05:15.190> on<00:05:15.310> and<00:05:15.460> so<00:05:15.610> forth<00:05:15.910> which<00:05:16.090> contain<00:05:16.540> other 00:05:16.980 --> 00:05:16.990 align:start position:0% so on and so forth which contain other 00:05:16.990 --> 00:05:20.010 align:start position:0% so on and so forth which contain other fields<00:05:17.500> and<00:05:18.190> then<00:05:18.370> may<00:05:18.610> statement<00:05:19.540> the<00:05:19.630> union 00:05:20.010 --> 00:05:20.020 align:start position:0% fields and then may statement the union 00:05:20.020 --> 00:05:23.820 align:start position:0% fields and then may statement the union of<00:05:20.229> all<00:05:20.440> these<00:05:20.650> statements<00:05:22.560> you<00:05:23.560> could<00:05:23.710> use 00:05:23.820 --> 00:05:23.830 align:start position:0% of all these statements you could use 00:05:23.830 --> 00:05:26.280 align:start position:0% of all these statements you could use the<00:05:23.979> keyword<00:05:24.340> one<00:05:24.580> off<00:05:24.790> to<00:05:25.000> make<00:05:25.180> a<00:05:25.419> union<00:05:25.930> of 00:05:26.280 --> 00:05:26.290 align:start position:0% the keyword one off to make a union of 00:05:26.290 --> 00:05:29.010 align:start position:0% the keyword one off to make a union of these<00:05:26.440> statements<00:05:27.000> so<00:05:28.000> essentially<00:05:28.330> with<00:05:28.840> the 00:05:29.010 --> 00:05:29.020 align:start position:0% these statements so essentially with the 00:05:29.020 --> 00:05:31.350 align:start position:0% these statements so essentially with the specification<00:05:29.770> of<00:05:29.949> the<00:05:30.190> fashion<00:05:30.760> until<00:05:31.210> you 00:05:31.350 --> 00:05:31.360 align:start position:0% specification of the fashion until you 00:05:31.360 --> 00:05:34.320 align:start position:0% specification of the fashion until you have<00:05:31.600> all<00:05:31.900> the<00:05:32.199> leaf<00:05:32.350> nodes<00:05:33.330> typically 00:05:34.320 --> 00:05:34.330 align:start position:0% have all the leaf nodes typically 00:05:34.330 --> 00:05:37.379 align:start position:0% have all the leaf nodes typically literals<00:05:35.169> or<00:05:36.000> constants<00:05:37.000> and<00:05:37.120> stuff<00:05:37.270> like 00:05:37.379 --> 00:05:37.389 align:start position:0% literals or constants and stuff like 00:05:37.389 --> 00:05:40.950 align:start position:0% literals or constants and stuff like that<00:05:37.600> and<00:05:38.639> yeah<00:05:39.639> essentially<00:05:40.389> try<00:05:40.660> to<00:05:40.690> cover 00:05:40.950 --> 00:05:40.960 align:start position:0% that and yeah essentially try to cover 00:05:40.960 --> 00:05:42.360 align:start position:0% that and yeah essentially try to cover as<00:05:41.260> many<00:05:41.410> aspects<00:05:41.889> of<00:05:41.950> the<00:05:42.040> programming 00:05:42.360 --> 00:05:42.370 align:start position:0% as many aspects of the programming 00:05:42.370 --> 00:05:43.440 align:start position:0% as many aspects of the programming language<00:05:42.729> as<00:05:42.820> possible 00:05:43.440 --> 00:05:43.450 align:start position:0% language as possible 00:05:43.450 --> 00:05:45.629 align:start position:0% language as possible well<00:05:43.720> mind<00:05:43.990> that<00:05:44.169> this<00:05:44.320> is<00:05:44.470> fully<00:05:44.740> handwritten 00:05:45.629 --> 00:05:45.639 align:start position:0% well mind that this is fully handwritten 00:05:45.639 --> 00:05:49.020 align:start position:0% well mind that this is fully handwritten certs<00:05:46.060> are<00:05:46.300> exhaustive<00:05:47.169> or<00:05:47.500> complete<00:05:47.889> but<00:05:48.669> for 00:05:49.020 --> 00:05:49.030 align:start position:0% certs are exhaustive or complete but for 00:05:49.030 --> 00:05:50.550 align:start position:0% certs are exhaustive or complete but for the<00:05:49.090> purpose<00:05:49.389> of<00:05:49.479> testing<00:05:49.720> the<00:05:50.200> hope<00:05:50.350> is<00:05:50.530> that 00:05:50.550 --> 00:05:50.560 align:start position:0% the purpose of testing the hope is that 00:05:50.560 --> 00:05:53.460 align:start position:0% the purpose of testing the hope is that it<00:05:50.889> covers<00:05:51.690> sufficient<00:05:52.690> language<00:05:53.020> features 00:05:53.460 --> 00:05:53.470 align:start position:0% it covers sufficient language features 00:05:53.470 --> 00:05:55.590 align:start position:0% it covers sufficient language features for<00:05:53.680> us<00:05:53.830> to<00:05:54.010> get<00:05:54.130> a<00:05:54.190> sufficient<00:05:54.610> of<00:05:54.940> assurance 00:05:55.590 --> 00:05:55.600 align:start position:0% for us to get a sufficient of assurance 00:05:55.600 --> 00:05:58.440 align:start position:0% for us to get a sufficient of assurance that<00:05:55.810> thing's<00:05:56.650> what<00:05:56.950> is<00:05:57.100> expected<00:05:57.370> of<00:05:57.790> course 00:05:58.440 --> 00:05:58.450 align:start position:0% that thing's what is expected of course 00:05:58.450 --> 00:06:01.110 align:start position:0% that thing's what is expected of course you<00:05:58.510> can<00:05:58.660> find<00:05:58.870> this<00:05:59.220> at<00:06:00.220> the<00:06:00.370> link<00:06:00.580> below<00:06:00.700> if 00:06:01.110 --> 00:06:01.120 align:start position:0% you can find this at the link below if 00:06:01.120 --> 00:06:04.710 align:start position:0% you can find this at the link below if you<00:06:01.270> are<00:06:01.300> interested<00:06:03.030> so<00:06:04.030> the<00:06:04.210> next<00:06:04.389> thing<00:06:04.600> is 00:06:04.710 --> 00:06:04.720 align:start position:0% you are interested so the next thing is 00:06:04.720 --> 00:06:06.990 align:start position:0% you are interested so the next thing is input<00:06:05.050> generation<00:06:05.530> we<00:06:06.010> have<00:06:06.039> aspect<00:06:06.639> how<00:06:06.940> do 00:06:06.990 --> 00:06:07.000 align:start position:0% input generation we have aspect how do 00:06:07.000 --> 00:06:11.850 align:start position:0% input generation we have aspect how do we<00:06:07.150> convert<00:06:07.539> respect<00:06:08.020> into<00:06:08.440> a<00:06:08.620> valid<00:06:08.919> input<00:06:10.860> we 00:06:11.850 --> 00:06:11.860 align:start position:0% we convert respect into a valid input we 00:06:11.860 --> 00:06:13.680 align:start position:0% we convert respect into a valid input we don't<00:06:12.100> generate<00:06:12.729> the<00:06:12.910> input<00:06:13.150> ourselves 00:06:13.680 --> 00:06:13.690 align:start position:0% don't generate the input ourselves 00:06:13.690 --> 00:06:15.840 align:start position:0% don't generate the input ourselves fortunately<00:06:14.380> there<00:06:15.070> is<00:06:15.100> a<00:06:15.190> library<00:06:15.639> called 00:06:15.840 --> 00:06:15.850 align:start position:0% fortunately there is a library called 00:06:15.850 --> 00:06:17.640 align:start position:0% fortunately there is a library called approval<00:06:16.389> of<00:06:16.539> your<00:06:16.750> data<00:06:16.930> which<00:06:17.229> is<00:06:17.380> also 00:06:17.640 --> 00:06:17.650 align:start position:0% approval of your data which is also 00:06:17.650 --> 00:06:20.190 align:start position:0% approval of your data which is also developed<00:06:18.070> at<00:06:18.220> Google<00:06:18.580> which<00:06:19.360> takes<00:06:19.840> the 00:06:20.190 --> 00:06:20.200 align:start position:0% developed at Google which takes the 00:06:20.200 --> 00:06:21.810 align:start position:0% developed at Google which takes the specification<00:06:21.039> shown<00:06:21.340> in<00:06:21.520> the<00:06:21.580> previous 00:06:21.810 --> 00:06:21.820 align:start position:0% specification shown in the previous 00:06:21.820 --> 00:06:25.700 align:start position:0% specification shown in the previous slide<00:06:21.910> and<00:06:22.650> converts<00:06:23.650> it<00:06:23.860> into<00:06:24.010> a<00:06:24.520> valid<00:06:24.820> input 00:06:25.700 --> 00:06:25.710 align:start position:0% slide and converts it into a valid input 00:06:25.710 --> 00:06:29.339 align:start position:0% slide and converts it into a valid input which<00:06:26.710> is<00:06:26.740> an<00:06:27.099> instantiation<00:06:27.460> of<00:06:28.060> the<00:06:28.240> spec<00:06:28.510> so 00:06:29.339 --> 00:06:29.349 align:start position:0% which is an instantiation of the spec so 00:06:29.349 --> 00:06:31.890 align:start position:0% which is an instantiation of the spec so each<00:06:29.800> input<00:06:30.160> is<00:06:30.430> essentially<00:06:30.820> free<00:06:31.210> for 00:06:31.890 --> 00:06:31.900 align:start position:0% each input is essentially free for 00:06:31.900 --> 00:06:35.490 align:start position:0% each input is essentially free for example<00:06:32.460> it<00:06:33.460> can<00:06:33.700> look<00:06:33.940> like<00:06:34.260> what<00:06:35.260> is<00:06:35.349> shown 00:06:35.490 --> 00:06:35.500 align:start position:0% example it can look like what is shown 00:06:35.500 --> 00:06:38.279 align:start position:0% example it can look like what is shown below<00:06:35.680> here<00:06:36.039> so<00:06:36.280> you<00:06:36.639> have<00:06:36.910> it<00:06:37.510> defines<00:06:37.900> blocks 00:06:38.279 --> 00:06:38.289 align:start position:0% below here so you have it defines blocks 00:06:38.289 --> 00:06:42.150 align:start position:0% below here so you have it defines blocks and<00:06:38.680> blocks<00:06:39.520> contain<00:06:40.030> some<00:06:40.270> diamond<00:06:41.130> which<00:06:42.130> is 00:06:42.150 --> 00:06:42.160 align:start position:0% and blocks contain some diamond which is 00:06:42.160 --> 00:06:44.700 align:start position:0% and blocks contain some diamond which is an<00:06:42.430> F<00:06:42.580> statement<00:06:42.760> in<00:06:43.090> this<00:06:43.150> case<00:06:43.210> and<00:06:43.780> if 00:06:44.700 --> 00:06:44.710 align:start position:0% an F statement in this case and if 00:06:44.710 --> 00:06:45.869 align:start position:0% an F statement in this case and if segment<00:06:44.979> has<00:06:45.310> a<00:06:45.340> condition 00:06:45.869 --> 00:06:45.879 align:start position:0% segment has a condition 00:06:45.879 --> 00:06:48.679 align:start position:0% segment has a condition which<00:06:46.599> contains<00:06:47.110> the<00:06:47.469> binary<00:06:47.800> operation 00:06:48.679 --> 00:06:48.689 align:start position:0% which contains the binary operation 00:06:48.689 --> 00:06:52.109 align:start position:0% which contains the binary operation which<00:06:49.689> is<00:06:49.869> an<00:06:49.990> equality<00:06:50.619> and<00:06:51.009> the<00:06:51.669> first<00:06:51.909> of 00:06:52.109 --> 00:06:52.119 align:start position:0% which is an equality and the first of 00:06:52.119 --> 00:06:54.359 align:start position:0% which is an equality and the first of operon<00:06:52.689> of<00:06:52.839> the<00:06:52.929> equalities<00:06:53.559> of<00:06:53.770> vertical 00:06:54.359 --> 00:06:54.369 align:start position:0% operon of the equalities of vertical 00:06:54.369 --> 00:06:57.809 align:start position:0% operon of the equalities of vertical reference<00:06:55.050> the<00:06:56.050> weibull's<00:06:56.469> ID<00:06:56.740> zero<00:06:57.219> and<00:06:57.580> then 00:06:57.809 --> 00:06:57.819 align:start position:0% reference the weibull's ID zero and then 00:06:57.819 --> 00:07:00.809 align:start position:0% reference the weibull's ID zero and then the<00:06:58.229> constant<00:06:59.229> x<00:06:59.619> it<00:06:59.919> is<00:06:59.979> being<00:07:00.369> compared 00:07:00.809 --> 00:07:00.819 align:start position:0% the constant x it is being compared 00:07:00.819 --> 00:07:05.570 align:start position:0% the constant x it is being compared against<00:07:01.240> is<00:07:01.509> 0<00:07:03.119> this<00:07:04.119> is<00:07:04.179> the<00:07:04.449> textual<00:07:04.809> form<00:07:05.199> of 00:07:05.570 --> 00:07:05.580 align:start position:0% against is 0 this is the textual form of 00:07:05.580 --> 00:07:09.869 align:start position:0% against is 0 this is the textual form of protobuf<00:07:06.580> message<00:07:07.199> for<00:07:08.199> clarity<00:07:08.619> but<00:07:08.889> of 00:07:09.869 --> 00:07:09.879 align:start position:0% protobuf message for clarity but of 00:07:09.879 --> 00:07:12.419 align:start position:0% protobuf message for clarity but of course<00:07:10.089> this<00:07:10.270> is<00:07:10.449> not<00:07:11.009> doesn't<00:07:12.009> really<00:07:12.189> make 00:07:12.419 --> 00:07:12.429 align:start position:0% course this is not doesn't really make 00:07:12.429 --> 00:07:14.730 align:start position:0% course this is not doesn't really make sense<00:07:12.459> yet<00:07:13.029> to<00:07:13.419> feed<00:07:13.659> this<00:07:13.839> to<00:07:14.110> this<00:07:14.259> to<00:07:14.499> the 00:07:14.730 --> 00:07:14.740 align:start position:0% sense yet to feed this to this to the 00:07:14.740 --> 00:07:19.499 align:start position:0% sense yet to feed this to this to the compiler<00:07:15.099> way<00:07:15.719> because<00:07:16.719> the<00:07:17.559> you<00:07:18.509> optimizer 00:07:19.499 --> 00:07:19.509 align:start position:0% compiler way because the you optimizer 00:07:19.509 --> 00:07:22.109 align:start position:0% compiler way because the you optimizer does<00:07:19.659> not<00:07:19.959> recognize<00:07:20.439> program<00:07:20.919> so<00:07:21.459> we<00:07:21.639> need<00:07:21.819> a 00:07:22.109 --> 00:07:22.119 align:start position:0% does not recognize program so we need a 00:07:22.119 --> 00:07:24.839 align:start position:0% does not recognize program so we need a program<00:07:22.599> that<00:07:22.779> converts<00:07:23.039> an<00:07:24.039> instance<00:07:24.490> of<00:07:24.610> the 00:07:24.839 --> 00:07:24.849 align:start position:0% program that converts an instance of the 00:07:24.849 --> 00:07:26.969 align:start position:0% program that converts an instance of the product<00:07:25.119> of<00:07:25.209> message<00:07:25.599> into<00:07:26.439> a<00:07:26.469> pilot<00:07:26.830> view 00:07:26.969 --> 00:07:26.979 align:start position:0% product of message into a pilot view 00:07:26.979 --> 00:07:29.249 align:start position:0% product of message into a pilot view program<00:07:27.399> and<00:07:27.789> this<00:07:27.939> is<00:07:28.059> where<00:07:28.269> the<00:07:28.719> converter 00:07:29.249 --> 00:07:29.259 align:start position:0% program and this is where the converter 00:07:29.259 --> 00:07:30.989 align:start position:0% program and this is where the converter program<00:07:29.499> comes<00:07:29.979> in<00:07:30.129> so<00:07:30.519> this<00:07:30.699> is<00:07:30.819> something 00:07:30.989 --> 00:07:30.999 align:start position:0% program comes in so this is something 00:07:30.999 --> 00:07:33.209 align:start position:0% program comes in so this is something that<00:07:31.179> we<00:07:31.749> have<00:07:32.019> to<00:07:32.199> write<00:07:32.349> but<00:07:32.589> fortunately 00:07:33.209 --> 00:07:33.219 align:start position:0% that we have to write but fortunately 00:07:33.219 --> 00:07:37.109 align:start position:0% that we have to write but fortunately this<00:07:33.519> is<00:07:33.669> not<00:07:33.699> too<00:07:33.939> complex<00:07:34.679> and<00:07:35.789> it's<00:07:36.789> part<00:07:36.999> of 00:07:37.109 --> 00:07:37.119 align:start position:0% this is not too complex and it's part of 00:07:37.119 --> 00:07:40.589 align:start position:0% this is not too complex and it's part of lines<00:07:37.449> of<00:07:37.569> code<00:07:38.369> so<00:07:39.369> essentially<00:07:39.699> converted 00:07:40.589 --> 00:07:40.599 align:start position:0% lines of code so essentially converted 00:07:40.599 --> 00:07:42.809 align:start position:0% lines of code so essentially converted as<00:07:40.659> a<00:07:40.719> source<00:07:40.990> to<00:07:41.139> source<00:07:41.319> translator<00:07:41.979> the 00:07:42.809 --> 00:07:42.819 align:start position:0% as a source to source translator the 00:07:42.819 --> 00:07:44.609 align:start position:0% as a source to source translator the input<00:07:43.179> is<00:07:43.330> the<00:07:43.360> product<00:07:43.749> of<00:07:43.839> serialization 00:07:44.609 --> 00:07:44.619 align:start position:0% input is the product of serialization 00:07:44.619 --> 00:07:49.429 align:start position:0% input is the product of serialization format<00:07:44.949> and<00:07:45.159> the<00:07:45.309> output<00:07:45.669> is<00:07:45.879> the<00:07:45.939> program<00:07:48.149> so 00:07:49.429 --> 00:07:49.439 align:start position:0% format and the output is the program so 00:07:49.439 --> 00:07:53.159 align:start position:0% format and the output is the program so we<00:07:50.439> talked<00:07:51.430> about<00:07:51.550> the<00:07:51.939> product<00:07:52.689> off<00:07:52.779> message 00:07:53.159 --> 00:07:53.169 align:start position:0% we talked about the product off message 00:07:53.169 --> 00:07:55.799 align:start position:0% we talked about the product off message maybe<00:07:53.979> previous<00:07:54.309> lyrics<00:07:54.669> how<00:07:55.149> it<00:07:55.300> looks<00:07:55.569> like 00:07:55.799 --> 00:07:55.809 align:start position:0% maybe previous lyrics how it looks like 00:07:55.809 --> 00:07:57.809 align:start position:0% maybe previous lyrics how it looks like when<00:07:56.229> you<00:07:56.349> convert<00:07:56.740> it<00:07:56.860> to<00:07:57.009> use<00:07:57.369> for<00:07:57.610> what<00:07:57.699> is 00:07:57.809 --> 00:07:57.819 align:start position:0% when you convert it to use for what is 00:07:57.819 --> 00:07:59.339 align:start position:0% when you convert it to use for what is shown<00:07:58.029> at<00:07:58.269> the<00:07:58.300> bottom<00:07:58.509> so<00:07:58.869> essentially<00:07:59.259> it's 00:07:59.339 --> 00:07:59.349 align:start position:0% shown at the bottom so essentially it's 00:07:59.349 --> 00:08:02.369 align:start position:0% shown at the bottom so essentially it's an<00:07:59.439> if<00:07:59.589> statement<00:07:59.939> with<00:08:00.939> available<00:08:01.779> called<00:08:02.019> X 00:08:02.369 --> 00:08:02.379 align:start position:0% an if statement with available called X 00:08:02.379 --> 00:08:05.639 align:start position:0% an if statement with available called X underscore<00:08:02.949> 0<00:08:03.490> if<00:08:03.939> it's<00:08:04.089> e<00:08:04.259> check<00:08:05.259> the<00:08:05.439> next 00:08:05.639 --> 00:08:05.649 align:start position:0% underscore 0 if it's e check the next 00:08:05.649 --> 00:08:09.719 align:start position:0% underscore 0 if it's e check the next equal<00:08:06.039> to<00:08:06.099> 0<00:08:07.019> of<00:08:08.019> course<00:08:08.229> this<00:08:08.409> is<00:08:08.559> a<00:08:08.769> snippet 00:08:09.719 --> 00:08:09.729 align:start position:0% equal to 0 of course this is a snippet 00:08:09.729 --> 00:08:12.389 align:start position:0% equal to 0 of course this is a snippet of<00:08:10.059> a<00:08:10.209> larger<00:08:10.569> piece<00:08:10.779> of<00:08:10.959> a<00:08:11.050> program<00:08:11.469> so<00:08:11.769> make 00:08:12.389 --> 00:08:12.399 align:start position:0% of a larger piece of a program so make 00:08:12.399 --> 00:08:15.540 align:start position:0% of a larger piece of a program so make sense<00:08:12.639> to<00:08:12.879> deposit<00:08:13.740> but<00:08:14.740> this<00:08:14.979> is<00:08:15.159> just<00:08:15.429> to 00:08:15.540 --> 00:08:15.550 align:start position:0% sense to deposit but this is just to 00:08:15.550 --> 00:08:18.079 align:start position:0% sense to deposit but this is just to give<00:08:15.699> you<00:08:15.819> an<00:08:15.879> idea<00:08:16.300> how<00:08:17.229> the<00:08:17.619> conversion 00:08:18.079 --> 00:08:18.089 align:start position:0% give you an idea how the conversion 00:08:18.089 --> 00:08:21.659 align:start position:0% give you an idea how the conversion looks<00:08:19.089> like<00:08:19.360> and<00:08:19.599> what<00:08:19.779> it<00:08:20.039> what<00:08:21.039> sticking<00:08:21.490> put 00:08:21.659 --> 00:08:21.669 align:start position:0% looks like and what it what sticking put 00:08:21.669 --> 00:08:25.529 align:start position:0% looks like and what it what sticking put any<00:08:21.909> output<00:08:22.209> of<00:08:22.929> the<00:08:23.050> conversion<00:08:23.529> process<00:08:24.539> but 00:08:25.529 --> 00:08:25.539 align:start position:0% any output of the conversion process but 00:08:25.539 --> 00:08:28.139 align:start position:0% any output of the conversion process but in<00:08:25.749> reality<00:08:25.929> we<00:08:26.409> have<00:08:26.439> the<00:08:26.800> complete<00:08:27.189> valid 00:08:28.139 --> 00:08:28.149 align:start position:0% in reality we have the complete valid 00:08:28.149 --> 00:08:33.760 align:start position:0% in reality we have the complete valid program<00:08:28.360> which<00:08:28.749> compiles<00:08:29.079> and<00:08:29.610> the<00:08:30.610> stopping 00:08:33.760 --> 00:08:33.770 align:start position:0% 00:08:33.770 --> 00:08:35.980 align:start position:0% to<00:08:34.340> put<00:08:34.550> these<00:08:34.730> two<00:08:34.940> pieces<00:08:34.970> together<00:08:35.450> what<00:08:35.810> we 00:08:35.980 --> 00:08:35.990 align:start position:0% to put these two pieces together what we 00:08:35.990 --> 00:08:38.710 align:start position:0% to put these two pieces together what we have<00:08:36.170> is<00:08:36.560> a<00:08:36.800> specification<00:08:37.360> that<00:08:38.360> we<00:08:38.570> have 00:08:38.710 --> 00:08:38.720 align:start position:0% have is a specification that we have 00:08:38.720 --> 00:08:41.860 align:start position:0% have is a specification that we have read<00:08:38.870> in<00:08:38.990> the<00:08:39.080> beginning<00:08:39.170> and<00:08:39.830> let<00:08:40.550> me<00:08:40.870> about 00:08:41.860 --> 00:08:41.870 align:start position:0% read in the beginning and let me about 00:08:41.870 --> 00:08:45.400 align:start position:0% read in the beginning and let me about fair<00:08:42.460> use<00:08:43.460> are<00:08:44.300> likely<00:08:44.570> to<00:08:44.690> generate<00:08:44.870> input 00:08:45.400 --> 00:08:45.410 align:start position:0% fair use are likely to generate input 00:08:45.410 --> 00:08:47.350 align:start position:0% fair use are likely to generate input but<00:08:45.680> that<00:08:45.860> input<00:08:46.040> is<00:08:46.310> not<00:08:46.460> ready<00:08:46.700> to<00:08:46.910> be<00:08:47.060> fed 00:08:47.350 --> 00:08:47.360 align:start position:0% but that input is not ready to be fed 00:08:47.360 --> 00:08:49.300 align:start position:0% but that input is not ready to be fed yes<00:08:47.570> because<00:08:47.780> it's<00:08:48.410> a<00:08:48.530> total<00:08:48.800> puff<00:08:48.920> language 00:08:49.300 --> 00:08:49.310 align:start position:0% yes because it's a total puff language 00:08:49.310 --> 00:08:52.930 align:start position:0% yes because it's a total puff language so<00:08:49.880> you<00:08:49.940> use<00:08:50.180> a<00:08:50.770> program<00:08:51.770> you<00:08:52.640> had<00:08:52.730> a<00:08:52.760> program 00:08:52.930 --> 00:08:52.940 align:start position:0% so you use a program you had a program 00:08:52.940 --> 00:08:54.760 align:start position:0% so you use a program you had a program called<00:08:53.150> for<00:08:53.900> a<00:08:53.930> buck<00:08:54.110> converter<00:08:54.290> which 00:08:54.760 --> 00:08:54.770 align:start position:0% called for a buck converter which 00:08:54.770 --> 00:08:57.190 align:start position:0% called for a buck converter which converts<00:08:54.800> from<00:08:55.340> this<00:08:55.610> language<00:08:56.030> to<00:08:56.720> a<00:08:56.960> valid 00:08:57.190 --> 00:08:57.200 align:start position:0% converts from this language to a valid 00:08:57.200 --> 00:08:59.710 align:start position:0% converts from this language to a valid test<00:08:57.470> program<00:08:57.920> that<00:08:58.340> can<00:08:58.490> then<00:08:58.640> be<00:08:58.700> fed<00:08:59.150> to<00:08:59.600> the 00:08:59.710 --> 00:08:59.720 align:start position:0% test program that can then be fed to the 00:08:59.720 --> 00:09:01.720 align:start position:0% test program that can then be fed to the compiler<00:09:00.050> so<00:09:00.890> finally<00:09:01.160> we<00:09:01.280> have<00:09:01.310> an<00:09:01.580> input 00:09:01.720 --> 00:09:01.730 align:start position:0% compiler so finally we have an input 00:09:01.730 --> 00:09:05.110 align:start position:0% compiler so finally we have an input that<00:09:02.030> could<00:09:02.210> be<00:09:02.360> used<00:09:02.720> to<00:09:02.930> test<00:09:03.680> my<00:09:03.980> lovely<00:09:04.400> but 00:09:05.110 --> 00:09:05.120 align:start position:0% that could be used to test my lovely but 00:09:05.120 --> 00:09:09.370 align:start position:0% that could be used to test my lovely but then<00:09:07.720> testing<00:09:08.720> the<00:09:08.810> compiler<00:09:09.230> actually 00:09:09.370 --> 00:09:09.380 align:start position:0% then testing the compiler actually 00:09:09.380 --> 00:09:11.710 align:start position:0% then testing the compiler actually requires<00:09:09.980> encoding<00:09:10.700> an<00:09:10.820> expectation<00:09:11.120> somehow 00:09:11.710 --> 00:09:11.720 align:start position:0% requires encoding an expectation somehow 00:09:11.720 --> 00:09:14.040 align:start position:0% requires encoding an expectation somehow so<00:09:11.930> imagine<00:09:12.470> that<00:09:12.500> you<00:09:12.980> randomly<00:09:13.370> test 00:09:14.040 --> 00:09:14.050 align:start position:0% so imagine that you randomly test 00:09:14.050 --> 00:09:16.870 align:start position:0% so imagine that you randomly test randomly<00:09:15.050> create<00:09:15.380> a<00:09:15.590> test<00:09:15.740> program<00:09:15.950> but<00:09:16.760> you 00:09:16.870 --> 00:09:16.880 align:start position:0% randomly create a test program but you 00:09:16.880 --> 00:09:18.100 align:start position:0% randomly create a test program but you don't<00:09:17.090> know<00:09:17.240> what<00:09:17.450> it's<00:09:17.600> supposed<00:09:17.660> to<00:09:17.960> be 00:09:18.100 --> 00:09:18.110 align:start position:0% don't know what it's supposed to be 00:09:18.110 --> 00:09:21.010 align:start position:0% don't know what it's supposed to be doing<00:09:18.290> what<00:09:18.680> side<00:09:18.860> effects<00:09:19.160> it<00:09:19.400> has<00:09:19.840> so<00:09:20.840> how<00:09:20.960> do 00:09:21.010 --> 00:09:21.020 align:start position:0% doing what side effects it has so how do 00:09:21.020 --> 00:09:23.650 align:start position:0% doing what side effects it has so how do you<00:09:21.170> encode<00:09:21.620> an<00:09:22.100> expectation<00:09:22.490> right<00:09:22.940> so<00:09:23.420> what 00:09:23.650 --> 00:09:23.660 align:start position:0% you encode an expectation right so what 00:09:23.660 --> 00:09:25.540 align:start position:0% you encode an expectation right so what do<00:09:23.720> you<00:09:23.780> check<00:09:24.020> how<00:09:24.860> do<00:09:24.920> you<00:09:25.010> check<00:09:25.220> out<00:09:25.340> that 00:09:25.540 --> 00:09:25.550 align:start position:0% do you check how do you check out that 00:09:25.550 --> 00:09:30.010 align:start position:0% do you check how do you check out that this<00:09:25.820> it's<00:09:26.180> doing<00:09:26.660> the<00:09:26.810> right<00:09:26.960> thing<00:09:29.020> the 00:09:30.010 --> 00:09:30.020 align:start position:0% this it's doing the right thing the 00:09:30.020 --> 00:09:31.540 align:start position:0% this it's doing the right thing the approach<00:09:30.290> that<00:09:30.530> we<00:09:30.680> use<00:09:30.830> as<00:09:31.040> differential 00:09:31.540 --> 00:09:31.550 align:start position:0% approach that we use as differential 00:09:31.550 --> 00:09:34.360 align:start position:0% approach that we use as differential fast<00:09:31.820> testing<00:09:32.230> so<00:09:33.230> essentially<00:09:33.680> in<00:09:34.010> it<00:09:34.160> worths 00:09:34.360 --> 00:09:34.370 align:start position:0% fast testing so essentially in it worths 00:09:34.370 --> 00:09:36.250 align:start position:0% fast testing so essentially in it worths tracking<00:09:34.850> the<00:09:34.970> side<00:09:35.180> effects<00:09:35.570> of<00:09:35.750> a<00:09:35.870> program 00:09:36.250 --> 00:09:36.260 align:start position:0% tracking the side effects of a program 00:09:36.260 --> 00:09:39.250 align:start position:0% tracking the side effects of a program using<00:09:37.190> an<00:09:37.400> execution<00:09:37.940> trace<00:09:38.050> running<00:09:39.050> the 00:09:39.250 --> 00:09:39.260 align:start position:0% using an execution trace running the 00:09:39.260 --> 00:09:41.110 align:start position:0% using an execution trace running the program<00:09:39.650> and<00:09:39.950> then<00:09:40.190> running<00:09:40.430> the<00:09:40.640> optimized 00:09:41.110 --> 00:09:41.120 align:start position:0% program and then running the optimized 00:09:41.120 --> 00:09:42.880 align:start position:0% program and then running the optimized version<00:09:41.360> of<00:09:41.630> the<00:09:41.690> program<00:09:41.840> and<00:09:42.230> comparing<00:09:42.770> the 00:09:42.880 --> 00:09:42.890 align:start position:0% version of the program and comparing the 00:09:42.890 --> 00:09:45.400 align:start position:0% version of the program and comparing the side<00:09:43.070> tracks<00:09:43.310> so<00:09:44.030> we<00:09:44.180> use<00:09:44.390> the<00:09:45.140> original 00:09:45.400 --> 00:09:45.410 align:start position:0% side tracks so we use the original 00:09:45.410 --> 00:09:49.350 align:start position:0% side tracks so we use the original program<00:09:46.010> as<00:09:46.220> a<00:09:46.280> baseline<00:09:47.410> to<00:09:48.410> compare<00:09:48.440> against 00:09:49.350 --> 00:09:49.360 align:start position:0% program as a baseline to compare against 00:09:49.360 --> 00:09:52.690 align:start position:0% program as a baseline to compare against and<00:09:50.410> we<00:09:51.410> compare<00:09:51.830> that<00:09:51.860> with<00:09:52.190> the<00:09:52.310> optimized 00:09:52.690 --> 00:09:52.700 align:start position:0% and we compare that with the optimized 00:09:52.700 --> 00:09:57.610 align:start position:0% and we compare that with the optimized program<00:09:52.940> so<00:09:53.420> it's<00:09:54.110> not<00:09:54.290> a<00:09:54.350> propagator<00:09:56.620> however 00:09:57.610 --> 00:09:57.620 align:start position:0% program so it's not a propagator however 00:09:57.620 --> 00:09:59.440 align:start position:0% program so it's not a propagator however before<00:09:57.860> we<00:09:58.070> can<00:09:58.160> do<00:09:58.460> that<00:09:58.730> we<00:09:59.060> need<00:09:59.240> an 00:09:59.440 --> 00:09:59.450 align:start position:0% before we can do that we need an 00:09:59.450 --> 00:10:01.660 align:start position:0% before we can do that we need an execution<00:10:00.020> quiz<00:10:00.260> which<00:10:00.620> track<00:10:00.950> side-effects 00:10:01.660 --> 00:10:01.670 align:start position:0% execution quiz which track side-effects 00:10:01.670 --> 00:10:04.510 align:start position:0% execution quiz which track side-effects of<00:10:01.820> the<00:10:02.510> execution<00:10:03.020> of<00:10:03.050> the<00:10:03.170> program<00:10:03.320> right<00:10:03.740> so 00:10:04.510 --> 00:10:04.520 align:start position:0% of the execution of the program right so 00:10:04.520 --> 00:10:06.610 align:start position:0% of the execution of the program right so we<00:10:05.000> need<00:10:05.120> to<00:10:05.360> know<00:10:05.900> somehow<00:10:06.320> what<00:10:06.530> is 00:10:06.610 --> 00:10:06.620 align:start position:0% we need to know somehow what is 00:10:06.620 --> 00:10:08.860 align:start position:0% we need to know somehow what is happening<00:10:07.070> to<00:10:07.610> check<00:10:07.880> whether<00:10:08.150> it<00:10:08.810> is 00:10:08.860 --> 00:10:08.870 align:start position:0% happening to check whether it is 00:10:08.870 --> 00:10:10.540 align:start position:0% happening to check whether it is happening<00:10:09.320> correctly<00:10:09.560> first<00:10:10.220> optimization 00:10:10.540 --> 00:10:10.550 align:start position:0% happening correctly first optimization 00:10:10.550 --> 00:10:14.830 align:start position:0% happening correctly first optimization and<00:10:11.420> this<00:10:12.110> is<00:10:12.230> where<00:10:12.790> the<00:10:13.790> Yui<00:10:13.940> interpreter 00:10:14.830 --> 00:10:14.840 align:start position:0% and this is where the Yui interpreter 00:10:14.840 --> 00:10:17.420 align:start position:0% and this is where the Yui interpreter causes<00:10:15.610> yeah 00:10:17.420 --> 00:10:17.430 align:start position:0% causes yeah 00:10:17.430 --> 00:10:18.860 align:start position:0% causes yeah essentially<00:10:18.030> an<00:10:18.120> interpreter<00:10:18.600> for<00:10:18.780> you 00:10:18.860 --> 00:10:18.870 align:start position:0% essentially an interpreter for you 00:10:18.870 --> 00:10:22.490 align:start position:0% essentially an interpreter for you programs<00:10:20.390> that<00:10:21.390> was<00:10:21.510> written<00:10:21.690> by<00:10:21.810> Chris 00:10:22.490 --> 00:10:22.500 align:start position:0% programs that was written by Chris 00:10:22.500 --> 00:10:24.380 align:start position:0% programs that was written by Chris so<00:10:22.740> essentially<00:10:23.010> what<00:10:23.220> it<00:10:23.340> does<00:10:23.370> is<00:10:23.700> it 00:10:24.380 --> 00:10:24.390 align:start position:0% so essentially what it does is it 00:10:24.390 --> 00:10:28.720 align:start position:0% so essentially what it does is it interprets<00:10:25.110> arbitrary<00:10:25.770> you<00:10:26.280> programs<00:10:26.840> so 00:10:28.720 --> 00:10:28.730 align:start position:0% interprets arbitrary you programs so 00:10:28.730 --> 00:10:32.540 align:start position:0% interprets arbitrary you programs so yeah<00:10:30.230> apart<00:10:31.230> from<00:10:31.320> interpretation<00:10:32.070> what<00:10:32.460> it 00:10:32.540 --> 00:10:32.550 align:start position:0% yeah apart from interpretation what it 00:10:32.550 --> 00:10:34.730 align:start position:0% yeah apart from interpretation what it additionally<00:10:33.210> does<00:10:33.420> is<00:10:33.660> outputs<00:10:34.380> the<00:10:34.530> side 00:10:34.730 --> 00:10:34.740 align:start position:0% additionally does is outputs the side 00:10:34.740 --> 00:10:37.280 align:start position:0% additionally does is outputs the side effect<00:10:35.070> of<00:10:35.250> the<00:10:35.430> program<00:10:35.580> as<00:10:36.060> a<00:10:36.120> place<00:10:36.480> a<00:10:36.780> trace 00:10:37.280 --> 00:10:37.290 align:start position:0% effect of the program as a place a trace 00:10:37.290 --> 00:10:39.560 align:start position:0% effect of the program as a place a trace can<00:10:37.500> be<00:10:37.590> thought<00:10:37.770> of<00:10:37.830> as<00:10:38.250> a<00:10:38.280> string<00:10:38.640> so<00:10:39.390> for 00:10:39.560 --> 00:10:39.570 align:start position:0% can be thought of as a string so for 00:10:39.570 --> 00:10:41.600 align:start position:0% can be thought of as a string so for example<00:10:40.050> you<00:10:40.500> have<00:10:40.650> the<00:10:40.800> test<00:10:40.980> program<00:10:41.160> under 00:10:41.600 --> 00:10:41.610 align:start position:0% example you have the test program under 00:10:41.610 --> 00:10:43.610 align:start position:0% example you have the test program under that<00:10:41.790> you<00:10:42.360> feel<00:10:42.600> it<00:10:42.720> to<00:10:42.810> the<00:10:42.900> yule<00:10:43.050> interpreter 00:10:43.610 --> 00:10:43.620 align:start position:0% that you feel it to the yule interpreter 00:10:43.620 --> 00:10:48.020 align:start position:0% that you feel it to the yule interpreter it<00:10:44.340> executes<00:10:45.300> it<00:10:46.130> step-by-step<00:10:47.130> and<00:10:47.640> then<00:10:47.910> in 00:10:48.020 --> 00:10:48.030 align:start position:0% it executes it step-by-step and then in 00:10:48.030 --> 00:10:49.970 align:start position:0% it executes it step-by-step and then in creates<00:10:48.420> this<00:10:48.600> execution<00:10:48.990> base<00:10:49.290> turn<00:10:49.740> on<00:10:49.890> the 00:10:49.970 --> 00:10:49.980 align:start position:0% creates this execution base turn on the 00:10:49.980 --> 00:10:52.250 align:start position:0% creates this execution base turn on the red<00:10:50.130> which<00:10:50.670> can<00:10:50.880> look<00:10:51.060> like<00:10:51.240> you<00:10:51.570> load 00:10:52.250 --> 00:10:52.260 align:start position:0% red which can look like you load 00:10:52.260 --> 00:10:54.590 align:start position:0% red which can look like you load something<00:10:52.530> from<00:10:53.220> memory<00:10:53.370> from<00:10:54.000> some<00:10:54.210> address 00:10:54.590 --> 00:10:54.600 align:start position:0% something from memory from some address 00:10:54.600 --> 00:10:57.470 align:start position:0% something from memory from some address X<00:10:54.930> and<00:10:55.320> then<00:10:55.440> store<00:10:55.770> it<00:10:55.800> so<00:10:56.700> the<00:10:56.880> slow<00:10:57.270> some 00:10:57.470 --> 00:10:57.480 align:start position:0% X and then store it so the slow some 00:10:57.480 --> 00:11:00.290 align:start position:0% X and then store it so the slow some value<00:10:57.780> and<00:10:58.080> I<00:10:58.290> might<00:10:58.800> do<00:10:59.460> need<00:10:59.670> a<00:10:59.700> copy<00:10:59.970> so<00:11:00.240> on 00:11:00.290 --> 00:11:00.300 align:start position:0% value and I might do need a copy so on 00:11:00.300 --> 00:11:02.480 align:start position:0% value and I might do need a copy so on and<00:11:00.450> so<00:11:00.570> forth<00:11:00.780> so<00:11:01.320> we<00:11:01.350> build<00:11:01.800> the<00:11:01.950> execution 00:11:02.480 --> 00:11:02.490 align:start position:0% and so forth so we build the execution 00:11:02.490 --> 00:11:05.420 align:start position:0% and so forth so we build the execution trace<00:11:02.610> of<00:11:03.090> the<00:11:03.660> test<00:11:03.870> program<00:11:04.310> using<00:11:05.310> the 00:11:05.420 --> 00:11:05.430 align:start position:0% trace of the test program using the 00:11:05.430 --> 00:11:09.020 align:start position:0% trace of the test program using the Yulin<00:11:05.730> doctor<00:11:07.040> and<00:11:08.040> finally<00:11:08.400> we<00:11:08.550> are<00:11:08.580> ready<00:11:08.820> to 00:11:09.020 --> 00:11:09.030 align:start position:0% Yulin doctor and finally we are ready to 00:11:09.030 --> 00:11:12.140 align:start position:0% Yulin doctor and finally we are ready to Hashmi<00:11:09.360> like<00:11:09.630> please<00:11:10.320> put<00:11:11.310> all<00:11:11.520> these<00:11:11.730> Doc's 00:11:12.140 --> 00:11:12.150 align:start position:0% Hashmi like please put all these Doc's 00:11:12.150 --> 00:11:16.160 align:start position:0% Hashmi like please put all these Doc's together<00:11:12.570> and<00:11:12.870> yes<00:11:13.560> the<00:11:14.070> optimizer<00:11:14.940> so<00:11:15.330> we 00:11:16.160 --> 00:11:16.170 align:start position:0% together and yes the optimizer so we 00:11:16.170 --> 00:11:18.770 align:start position:0% together and yes the optimizer so we suck<00:11:16.350> returning<00:11:16.920> the<00:11:17.120> program<00:11:18.120> feed<00:11:18.510> it<00:11:18.630> to 00:11:18.770 --> 00:11:18.780 align:start position:0% suck returning the program feed it to 00:11:18.780 --> 00:11:20.690 align:start position:0% suck returning the program feed it to the<00:11:18.900> interpreter<00:11:19.370> optimize<00:11:20.370> the<00:11:20.490> same 00:11:20.690 --> 00:11:20.700 align:start position:0% the interpreter optimize the same 00:11:20.700 --> 00:11:23.210 align:start position:0% the interpreter optimize the same program<00:11:21.120> pizza<00:11:21.750> again<00:11:22.320> the<00:11:22.800> optimized 00:11:23.210 --> 00:11:23.220 align:start position:0% program pizza again the optimized 00:11:23.220 --> 00:11:25.400 align:start position:0% program pizza again the optimized version<00:11:23.520> to<00:11:23.850> the<00:11:23.910> interpreter<00:11:24.540> and<00:11:24.960> then<00:11:25.290> you 00:11:25.400 --> 00:11:25.410 align:start position:0% version to the interpreter and then you 00:11:25.410 --> 00:11:27.560 align:start position:0% version to the interpreter and then you get<00:11:25.560> to<00:11:25.830> execution<00:11:26.400> fixes<00:11:26.820> with<00:11:27.120> essentially 00:11:27.560 --> 00:11:27.570 align:start position:0% get to execution fixes with essentially 00:11:27.570 --> 00:11:30.400 align:start position:0% get to execution fixes with essentially strings<00:11:27.900> and<00:11:28.200> then<00:11:28.590> we<00:11:28.710> can<00:11:28.860> simply<00:11:29.160> do<00:11:29.280> string 00:11:30.400 --> 00:11:30.410 align:start position:0% strings and then we can simply do string 00:11:30.410 --> 00:11:34.250 align:start position:0% strings and then we can simply do string equality<00:11:31.410> check<00:11:31.590> right<00:11:31.830> so<00:11:32.990> the<00:11:33.990> execution 00:11:34.250 --> 00:11:34.260 align:start position:0% equality check right so the execution 00:11:34.260 --> 00:11:37.190 align:start position:0% equality check right so the execution traces<00:11:34.920> of<00:11:35.160> what<00:11:35.880> these<00:11:36.060> versions<00:11:36.330> are<00:11:36.720> equal 00:11:37.190 --> 00:11:37.200 align:start position:0% traces of what these versions are equal 00:11:37.200 --> 00:11:39.620 align:start position:0% traces of what these versions are equal everything's<00:11:37.770> fine<00:11:38.040> as<00:11:38.370> we<00:11:38.580> expect<00:11:39.030> if<00:11:39.480> it's 00:11:39.620 --> 00:11:39.630 align:start position:0% everything's fine as we expect if it's 00:11:39.630 --> 00:11:42.200 align:start position:0% everything's fine as we expect if it's not<00:11:39.810> equal<00:11:39.930> it<00:11:40.230> has<00:11:40.320> a<00:11:40.350> part<00:11:40.650> most<00:11:41.640> likely<00:11:42.030> in 00:11:42.200 --> 00:11:42.210 align:start position:0% not equal it has a part most likely in 00:11:42.210 --> 00:11:44.150 align:start position:0% not equal it has a part most likely in the<00:11:42.270> optimizer<00:11:42.690> but<00:11:43.350> in<00:11:43.740> practice<00:11:44.130> people 00:11:44.150 --> 00:11:44.160 align:start position:0% the optimizer but in practice people 00:11:44.160 --> 00:11:46.070 align:start position:0% the optimizer but in practice people also<00:11:44.490> had<00:11:44.790> situations<00:11:45.060> where<00:11:45.480> the<00:11:45.690> bug<00:11:45.840> is<00:11:45.960> a 00:11:46.070 --> 00:11:46.080 align:start position:0% also had situations where the bug is a 00:11:46.080 --> 00:11:52.090 align:start position:0% also had situations where the bug is a per<00:11:46.140> se<00:11:46.380> but<00:11:47.010> with<00:11:47.850> high<00:11:48.090> likelihood<00:11:48.630> it's<00:11:49.080> do 00:11:52.090 --> 00:11:52.100 align:start position:0% 00:11:52.100 --> 00:11:54.530 align:start position:0% so<00:11:53.100> essentially<00:11:53.730> like<00:11:53.910> I<00:11:53.940> said<00:11:54.210> we<00:11:54.360> found 00:11:54.530 --> 00:11:54.540 align:start position:0% so essentially like I said we found 00:11:54.540 --> 00:11:57.020 align:start position:0% so essentially like I said we found about<00:11:54.840> this<00:11:55.410> process<00:11:55.890> be<00:11:56.130> pretty<00:11:56.760> effective 00:11:57.020 --> 00:11:57.030 align:start position:0% about this process be pretty effective 00:11:57.030 --> 00:11:59.270 align:start position:0% about this process be pretty effective in<00:11:57.330> practice<00:11:57.510> we<00:11:58.290> found<00:11:58.530> about<00:11:58.710> seven<00:11:59.070> bucks 00:11:59.270 --> 00:11:59.280 align:start position:0% in practice we found about seven bucks 00:11:59.280 --> 00:12:02.600 align:start position:0% in practice we found about seven bucks dope<00:11:59.730> which<00:12:00.000> was<00:12:00.810> in<00:12:01.110> the<00:12:01.350> optimization 00:12:02.600 --> 00:12:02.610 align:start position:0% dope which was in the optimization 00:12:02.610 --> 00:12:05.630 align:start position:0% dope which was in the optimization rule<00:12:03.240> that<00:12:03.810> was<00:12:04.320> used<00:12:04.590> to<00:12:04.800> essentially 00:12:05.630 --> 00:12:05.640 align:start position:0% rule that was used to essentially 00:12:05.640 --> 00:12:07.700 align:start position:0% rule that was used to essentially optimize<00:12:06.060> programs<00:12:06.450> fiber<00:12:07.290> in<00:12:07.590> the 00:12:07.700 --> 00:12:07.710 align:start position:0% optimize programs fiber in the 00:12:07.710 --> 00:12:09.620 align:start position:0% optimize programs fiber in the experiment<00:12:08.220> he<00:12:08.310> you<00:12:08.520> love<00:12:08.670> to<00:12:08.790> myself<00:12:08.970> yeah 00:12:09.620 --> 00:12:09.630 align:start position:0% experiment he you love to myself yeah 00:12:09.630 --> 00:12:11.610 align:start position:0% experiment he you love to myself yeah it's<00:12:09.990> not<00:12:10.200> supposed<00:12:10.380> to<00:12:10.650> be<00:12:10.800> used<00:12:10.980> by 00:12:11.610 --> 00:12:11.620 align:start position:0% it's not supposed to be used by 00:12:11.620 --> 00:12:15.060 align:start position:0% it's not supposed to be used by right<00:12:11.770> now<00:12:12.330> so<00:12:13.330> that<00:12:14.200> suppose<00:12:14.530> of<00:12:14.710> passing 00:12:15.060 --> 00:12:15.070 align:start position:0% right now so that suppose of passing 00:12:15.070 --> 00:12:22.200 align:start position:0% right now so that suppose of passing anyway<00:12:16.440> so<00:12:18.510> yeah<00:12:19.510> -<00:12:19.750> in<00:12:20.170> the<00:12:20.520> EBM<00:12:21.520> optimizes 00:12:22.200 --> 00:12:22.210 align:start position:0% anyway so yeah - in the EBM optimizes 00:12:22.210 --> 00:12:24.690 align:start position:0% anyway so yeah - in the EBM optimizes pellucid<00:12:22.960> production<00:12:23.410> but<00:12:23.620> fortunately<00:12:24.430> he's 00:12:24.690 --> 00:12:24.700 align:start position:0% pellucid production but fortunately he's 00:12:24.700 --> 00:12:30.480 align:start position:0% pellucid production but fortunately he's low<00:12:25.240> very<00:12:25.570> low<00:12:25.660> severity<00:12:27.780> yeah<00:12:29.490> mostly 00:12:30.480 --> 00:12:30.490 align:start position:0% low very low severity yeah mostly 00:12:30.490 --> 00:12:34.680 align:start position:0% low very low severity yeah mostly because<00:12:31.110> the<00:12:32.280> the<00:12:33.280> bug<00:12:33.790> equivalent<00:12:34.270> question 00:12:34.680 --> 00:12:34.690 align:start position:0% because the the bug equivalent question 00:12:34.690 --> 00:12:41.610 align:start position:0% because the the bug equivalent question was<00:12:35.590> optimizing<00:12:36.430> Boston<00:12:37.000> so<00:12:37.710> it<00:12:38.710> was<00:12:39.690> it<00:12:40.690> was<00:12:41.050> a 00:12:41.610 --> 00:12:41.620 align:start position:0% was optimizing Boston so it was it was a 00:12:41.620 --> 00:12:43.590 align:start position:0% was optimizing Boston so it was it was a very<00:12:42.070> specific<00:12:42.340> pattern<00:12:42.820> that<00:12:43.180> was<00:12:43.360> going 00:12:43.590 --> 00:12:43.600 align:start position:0% very specific pattern that was going 00:12:43.600 --> 00:12:46.170 align:start position:0% very specific pattern that was going wrong<00:12:43.630> and<00:12:44.020> this<00:12:44.200> pattern<00:12:44.460> could<00:12:45.460> be<00:12:45.520> detected 00:12:46.170 --> 00:12:46.180 align:start position:0% wrong and this pattern could be detected 00:12:46.180 --> 00:12:48.240 align:start position:0% wrong and this pattern could be detected visually<00:12:46.360> because<00:12:46.930> it's<00:12:47.080> a<00:12:47.770> compile-time 00:12:48.240 --> 00:12:48.250 align:start position:0% visually because it's a compile-time 00:12:48.250 --> 00:12:52.140 align:start position:0% visually because it's a compile-time constant<00:12:48.460> or<00:12:49.830> yes<00:12:50.830> that's<00:12:51.520> essentially<00:12:51.970> why 00:12:52.140 --> 00:12:52.150 align:start position:0% constant or yes that's essentially why 00:12:52.150 --> 00:12:55.620 align:start position:0% constant or yes that's essentially why it<00:12:52.630> was<00:12:52.780> low<00:12:53.220> the<00:12:54.220> others<00:12:54.460> well<00:12:54.700> of<00:12:55.060> course<00:12:55.120> in 00:12:55.620 --> 00:12:55.630 align:start position:0% it was low the others well of course in 00:12:55.630 --> 00:12:58.860 align:start position:0% it was low the others well of course in the<00:12:55.870> experimental<00:12:56.380> version<00:12:56.860> so<00:12:57.720> that's<00:12:58.720> why 00:12:58.860 --> 00:12:58.870 align:start position:0% the experimental version so that's why 00:12:58.870 --> 00:13:03.030 align:start position:0% the experimental version so that's why we're<00:12:59.050> testing<00:12:59.200> it<00:13:01.290> so<00:13:02.290> that's<00:13:02.470> what<00:13:02.620> we've 00:13:03.030 --> 00:13:03.040 align:start position:0% we're testing it so that's what we've 00:13:03.040 --> 00:13:03.270 align:start position:0% we're testing it so that's what we've done 00:13:03.270 --> 00:13:03.280 align:start position:0% done 00:13:03.280 --> 00:13:06.630 align:start position:0% done channel<00:13:03.940> is<00:13:04.030> to<00:13:04.180> be<00:13:04.330> going<00:13:04.690> forward<00:13:05.460> the<00:13:06.460> main 00:13:06.630 --> 00:13:06.640 align:start position:0% channel is to be going forward the main 00:13:06.640 --> 00:13:08.790 align:start position:0% channel is to be going forward the main thing<00:13:06.820> is<00:13:06.940> we<00:13:07.300> would<00:13:07.480> like<00:13:07.510> to<00:13:07.690> find<00:13:08.260> high 00:13:08.790 --> 00:13:08.800 align:start position:0% thing is we would like to find high 00:13:08.800 --> 00:13:10.440 align:start position:0% thing is we would like to find high severity<00:13:09.130> bugs<00:13:09.520> before<00:13:09.730> the<00:13:10.120> production 00:13:10.440 --> 00:13:10.450 align:start position:0% severity bugs before the production 00:13:10.450 --> 00:13:13.590 align:start position:0% severity bugs before the production right<00:13:11.020> because<00:13:11.370> that's<00:13:12.370> what<00:13:12.580> matters<00:13:12.910> the 00:13:13.590 --> 00:13:13.600 align:start position:0% right because that's what matters the 00:13:13.600 --> 00:13:16.380 align:start position:0% right because that's what matters the main<00:13:13.960> problem<00:13:14.350> with<00:13:14.380> fuzzing<00:13:15.190> a<00:13:15.370> compiler<00:13:15.760> for 00:13:16.380 --> 00:13:16.390 align:start position:0% main problem with fuzzing a compiler for 00:13:16.390 --> 00:13:19.560 align:start position:0% main problem with fuzzing a compiler for correctness<00:13:16.750> is<00:13:17.110> that<00:13:17.500> it's<00:13:17.830> usually<00:13:18.570> slow 00:13:19.560 --> 00:13:19.570 align:start position:0% correctness is that it's usually slow 00:13:19.570 --> 00:13:21.270 align:start position:0% correctness is that it's usually slow it's<00:13:19.780> a<00:13:19.870> slow<00:13:20.080> process<00:13:20.260> I<00:13:20.530> mean<00:13:20.740> typically<00:13:21.070> in 00:13:21.270 --> 00:13:21.280 align:start position:0% it's a slow process I mean typically in 00:13:21.280 --> 00:13:23.720 align:start position:0% it's a slow process I mean typically in fuzzing<00:13:21.610> you<00:13:22.480> select<00:13:22.810> the<00:13:22.900> small<00:13:23.170> piece<00:13:23.350> of 00:13:23.720 --> 00:13:23.730 align:start position:0% fuzzing you select the small piece of 00:13:23.730 --> 00:13:26.460 align:start position:0% fuzzing you select the small piece of code<00:13:24.730> that<00:13:24.760> security<00:13:25.360> critical<00:13:25.750> and<00:13:25.990> deposit 00:13:26.460 --> 00:13:26.470 align:start position:0% code that security critical and deposit 00:13:26.470 --> 00:13:29.130 align:start position:0% code that security critical and deposit which<00:13:26.740> means<00:13:27.090> typically<00:13:28.090> you<00:13:28.810> would<00:13:28.960> like 00:13:29.130 --> 00:13:29.140 align:start position:0% which means typically you would like 00:13:29.140 --> 00:13:32.100 align:start position:0% which means typically you would like execution<00:13:29.980> speeds<00:13:30.250> of<00:13:30.400> over<00:13:30.580> 100<00:13:31.240> per<00:13:31.750> second 00:13:32.100 --> 00:13:32.110 align:start position:0% execution speeds of over 100 per second 00:13:32.110 --> 00:13:34.580 align:start position:0% execution speeds of over 100 per second but<00:13:32.920> for<00:13:33.190> example<00:13:33.280> if<00:13:33.670> you<00:13:33.760> want<00:13:33.970> to<00:13:34.030> test 00:13:34.580 --> 00:13:34.590 align:start position:0% but for example if you want to test 00:13:34.590 --> 00:13:38.760 align:start position:0% but for example if you want to test component<00:13:36.060> resulted<00:13:37.060> step2<00:13:37.990> encoder<00:13:38.440> inside 00:13:38.760 --> 00:13:38.770 align:start position:0% component resulted step2 encoder inside 00:13:38.770 --> 00:13:42.900 align:start position:0% component resulted step2 encoder inside solidity<00:13:40.320> the<00:13:41.320> problem<00:13:41.620> is<00:13:41.740> that<00:13:42.010> compilation 00:13:42.900 --> 00:13:42.910 align:start position:0% solidity the problem is that compilation 00:13:42.910 --> 00:13:45.510 align:start position:0% solidity the problem is that compilation is<00:13:43.150> slow<00:13:43.510> and<00:13:43.780> this<00:13:44.350> is<00:13:44.500> perfectly<00:13:44.980> fine<00:13:45.220> for 00:13:45.510 --> 00:13:45.520 align:start position:0% is slow and this is perfectly fine for 00:13:45.520 --> 00:13:47.820 align:start position:0% is slow and this is perfectly fine for the<00:13:45.580> use<00:13:45.730> case<00:13:45.970> because<00:13:46.590> developers<00:13:47.590> can 00:13:47.820 --> 00:13:47.830 align:start position:0% the use case because developers can 00:13:47.830 --> 00:13:50.370 align:start position:0% the use case because developers can spend<00:13:48.160> an<00:13:48.280> additional<00:13:48.400> second<00:13:49.060> or<00:13:49.180> so<00:13:49.360> to<00:13:49.630> save 00:13:50.370 --> 00:13:50.380 align:start position:0% spend an additional second or so to save 00:13:50.380 --> 00:13:51.810 align:start position:0% spend an additional second or so to save gas<00:13:50.680> right 00:13:51.810 --> 00:13:51.820 align:start position:0% gas right 00:13:51.820 --> 00:13:54.510 align:start position:0% gas right the<00:13:52.240> only<00:13:52.390> problem<00:13:52.780> is<00:13:52.870> that<00:13:52.930> if<00:13:53.260> you<00:13:53.520> apply 00:13:54.510 --> 00:13:54.520 align:start position:0% the only problem is that if you apply 00:13:54.520 --> 00:13:57.660 align:start position:0% the only problem is that if you apply for<00:13:54.930> processing<00:13:55.930> it<00:13:56.170> it<00:13:56.980> can<00:13:57.220> become<00:13:57.550> a 00:13:57.660 --> 00:13:57.670 align:start position:0% for processing it it can become a 00:13:57.670 --> 00:14:03.570 align:start position:0% for processing it it can become a bottleneck<00:13:57.930> so<00:14:00.630> to<00:14:01.630> find<00:14:01.930> ways<00:14:02.200> to<00:14:02.590> basically 00:14:03.570 --> 00:14:03.580 align:start position:0% bottleneck so to find ways to basically 00:14:03.580 --> 00:14:06.020 align:start position:0% bottleneck so to find ways to basically make<00:14:04.210> it<00:14:04.330> more<00:14:04.480> suitable<00:14:04.870> to<00:14:04.960> fuzzing<00:14:05.380> is 00:14:06.020 --> 00:14:06.030 align:start position:0% make it more suitable to fuzzing is 00:14:06.030 --> 00:14:09.680 align:start position:0% make it more suitable to fuzzing is charge<00:14:07.030> that<00:14:07.360> we<00:14:07.570> have<00:14:07.660> currently<00:14:07.750> working<00:14:08.170> on 00:14:09.680 --> 00:14:09.690 align:start position:0% charge that we have currently working on 00:14:09.690 --> 00:14:13.319 align:start position:0% charge that we have currently working on so<00:14:10.690> in<00:14:10.960> conclusion<00:14:11.140> we<00:14:12.100> started<00:14:12.430> doing<00:14:12.520> it 00:14:13.319 --> 00:14:13.329 align:start position:0% so in conclusion we started doing it 00:14:13.329 --> 00:14:15.389 align:start position:0% so in conclusion we started doing it structure<00:14:13.660> we're<00:14:13.899> pausing<00:14:14.290> to<00:14:14.559> detect 00:14:15.389 --> 00:14:15.399 align:start position:0% structure we're pausing to detect 00:14:15.399 --> 00:14:18.239 align:start position:0% structure we're pausing to detect problems<00:14:15.639> so<00:14:15.999> the<00:14:16.149> optimizer<00:14:16.629> and<00:14:17.069> alert<00:14:18.069> us 00:14:18.239 --> 00:14:18.249 align:start position:0% problems so the optimizer and alert us 00:14:18.249 --> 00:14:21.949 align:start position:0% problems so the optimizer and alert us whenever<00:14:18.819> there's<00:14:19.089> a<00:14:19.149> part<00:14:19.389> in<00:14:19.660> the<00:14:20.519> code<00:14:21.519> base 00:14:21.949 --> 00:14:21.959 align:start position:0% whenever there's a part in the code base 00:14:21.959 --> 00:14:25.139 align:start position:0% whenever there's a part in the code base it<00:14:22.959> has<00:14:23.259> been<00:14:23.709> so<00:14:23.920> far<00:14:24.100> is<00:14:24.220> used<00:14:24.610> for<00:14:24.970> me 00:14:25.139 --> 00:14:25.149 align:start position:0% it has been so far is used for me 00:14:25.149 --> 00:14:27.869 align:start position:0% it has been so far is used for me testing<00:14:25.809> the<00:14:25.929> optimizer<00:14:26.410> and<00:14:26.799> leader<00:14:27.009> and 00:14:27.869 --> 00:14:27.879 align:start position:0% testing the optimizer and leader and 00:14:27.879 --> 00:14:30.929 align:start position:0% testing the optimizer and leader and then<00:14:28.119> decoding<00:14:28.920> it<00:14:29.920> has<00:14:30.100> decent<00:14:30.309> as<00:14:30.579> Charles 00:14:30.929 --> 00:14:30.939 align:start position:0% then decoding it has decent as Charles 00:14:30.939 --> 00:14:33.329 align:start position:0% then decoding it has decent as Charles but<00:14:31.269> Ben<00:14:31.660> mind<00:14:31.929> that<00:14:32.139> testing<00:14:32.499> is<00:14:32.559> not<00:14:32.739> formal 00:14:33.329 --> 00:14:33.339 align:start position:0% but Ben mind that testing is not formal 00:14:33.339 --> 00:14:35.249 align:start position:0% but Ben mind that testing is not formal it<00:14:33.879> doesn't<00:14:34.569> give<00:14:34.660> you<00:14:34.779> any<00:14:34.839> formal 00:14:35.249 --> 00:14:35.259 align:start position:0% it doesn't give you any formal 00:14:35.259 --> 00:14:36.449 align:start position:0% it doesn't give you any formal guarantees 00:14:36.449 --> 00:14:36.459 align:start position:0% guarantees 00:14:36.459 --> 00:14:38.189 align:start position:0% guarantees yeah<00:14:37.059> take<00:14:37.360> that<00:14:37.389> with<00:14:37.660> the<00:14:37.749> green<00:14:37.929> sauce 00:14:38.189 --> 00:14:38.199 align:start position:0% yeah take that with the green sauce 00:14:38.199 --> 00:14:41.789 align:start position:0% yeah take that with the green sauce right<00:14:38.559> yeah<00:14:39.239> that's<00:14:40.239> about<00:14:40.420> it<00:14:40.689> yeah<00:14:41.049> thank 00:14:41.789 --> 00:14:41.799 align:start position:0% right yeah that's about it yeah thank 00:14:41.799 --> 00:14:55.169 align:start position:0% right yeah that's about it yeah thank you<00:14:52.619> so<00:14:53.619> how<00:14:53.769> much<00:14:53.889> do<00:14:54.069> you<00:14:54.160> know<00:14:54.309> about<00:14:54.339> the 00:14:55.169 --> 00:14:55.179 align:start position:0% you so how much do you know about the 00:14:55.179 --> 00:14:57.229 align:start position:0% you so how much do you know about the coverage<00:14:55.480> the<00:14:55.689> agreements<00:14:56.079> again<00:14:56.319> with<00:14:56.739> say 00:14:57.229 --> 00:14:57.239 align:start position:0% coverage the agreements again with say 00:14:57.239 --> 00:15:02.160 align:start position:0% coverage the agreements again with say one<00:14:58.239> day<00:14:58.389> a<00:14:58.420> person<00:14:59.939> but<00:15:00.939> one<00:15:01.299> day<00:15:01.600> it's<00:15:01.989> not 00:15:02.160 --> 00:15:02.170 align:start position:0% one day a person but one day it's not 00:15:02.170 --> 00:15:04.079 align:start position:0% one day a person but one day it's not safer<00:15:02.649> with<00:15:03.040> the<00:15:03.129> society<00:15:03.459> running<00:15:03.759> every<00:15:03.970> day 00:15:04.079 --> 00:15:04.089 align:start position:0% safer with the society running every day 00:15:04.089 --> 00:15:06.509 align:start position:0% safer with the society running every day right<00:15:04.329> answer<00:15:04.600> but<00:15:05.139> then<00:15:05.499> it<00:15:05.709> bits<00:15:06.040> an<00:15:06.189> office 00:15:06.509 --> 00:15:06.519 align:start position:0% right answer but then it bits an office 00:15:06.519 --> 00:15:10.379 align:start position:0% right answer but then it bits an office over<00:15:06.759> time<00:15:06.970> so<00:15:07.600> we<00:15:07.720> started<00:15:08.110> and<00:15:08.319> then<00:15:09.239> bits<00:15:10.239> of 00:15:10.379 --> 00:15:10.389 align:start position:0% over time so we started and then bits of 00:15:10.389 --> 00:15:12.059 align:start position:0% over time so we started and then bits of corpus<00:15:10.720> and<00:15:10.869> then<00:15:11.019> the<00:15:11.230> same<00:15:11.410> corpus<00:15:11.769> is<00:15:11.889> used 00:15:12.059 --> 00:15:12.069 align:start position:0% corpus and then the same corpus is used 00:15:12.069 --> 00:15:14.519 align:start position:0% corpus and then the same corpus is used so<00:15:12.699> you<00:15:13.059> can<00:15:13.209> think<00:15:13.480> of<00:15:13.629> it<00:15:13.809> as<00:15:13.839> a<00:15:13.989> cumulative 00:15:14.519 --> 00:15:14.529 align:start position:0% so you can think of it as a cumulative 00:15:14.529 --> 00:15:16.650 align:start position:0% so you can think of it as a cumulative curve<00:15:14.949> it<00:15:15.369> improves<00:15:15.730> over<00:15:16.119> time<00:15:16.149> and<00:15:16.509> then 00:15:16.650 --> 00:15:16.660 align:start position:0% curve it improves over time and then 00:15:16.660 --> 00:15:18.840 align:start position:0% curve it improves over time and then keeps<00:15:17.410> increasing<00:15:17.619> essentially<00:15:18.309> so 00:15:18.840 --> 00:15:18.850 align:start position:0% keeps increasing essentially so 00:15:18.850 --> 00:15:21.689 align:start position:0% keeps increasing essentially so essentially<00:15:19.239> right<00:15:19.569> now<00:15:19.779> as<00:15:19.959> it<00:15:20.110> stands<00:15:20.699> just 00:15:21.689 --> 00:15:21.699 align:start position:0% essentially right now as it stands just 00:15:21.699 --> 00:15:23.879 align:start position:0% essentially right now as it stands just for<00:15:21.879> the<00:15:21.970> optimizer<00:15:22.480> it's<00:15:22.720> pretty<00:15:23.019> good<00:15:23.169> it's 00:15:23.879 --> 00:15:23.889 align:start position:0% for the optimizer it's pretty good it's 00:15:23.889 --> 00:15:25.829 align:start position:0% for the optimizer it's pretty good it's about<00:15:24.040> ninety<00:15:24.730> eight<00:15:24.910> ninety<00:15:24.939> three<00:15:25.660> percent 00:15:25.829 --> 00:15:25.839 align:start position:0% about ninety eight ninety three percent 00:15:25.839 --> 00:15:30.720 align:start position:0% about ninety eight ninety three percent something<00:15:26.319> of<00:15:26.980> edge<00:15:27.459> coverage<00:15:27.929> yeah<00:15:29.610> and<00:15:30.610> we 00:15:30.720 --> 00:15:30.730 align:start position:0% something of edge coverage yeah and we 00:15:30.730 --> 00:15:32.789 align:start position:0% something of edge coverage yeah and we keep<00:15:30.879> running<00:15:31.660> keep<00:15:31.869> an<00:15:32.019> eye<00:15:32.139> on<00:15:32.169> it<00:15:32.439> and<00:15:32.619> then 00:15:32.789 --> 00:15:32.799 align:start position:0% keep running keep an eye on it and then 00:15:32.799 --> 00:15:34.519 align:start position:0% keep running keep an eye on it and then see<00:15:33.309> if<00:15:33.549> we<00:15:33.759> can 00:15:34.519 --> 00:15:34.529 align:start position:0% see if we can 00:15:34.529 --> 00:15:37.829 align:start position:0% see if we can they<00:15:35.529> idea<00:15:36.069> what's<00:15:36.670> missing<00:15:37.089> is<00:15:37.209> it<00:15:37.360> is<00:15:37.629> it 00:15:37.829 --> 00:15:37.839 align:start position:0% they idea what's missing is it is it 00:15:37.839 --> 00:15:39.480 align:start position:0% they idea what's missing is it is it syntactic<00:15:38.559> features<00:15:38.769> at<00:15:39.160> the<00:15:39.249> end<00:15:39.369> of<00:15:39.429> the 00:15:39.480 --> 00:15:39.490 align:start position:0% syntactic features at the end of the 00:15:39.490 --> 00:15:40.919 align:start position:0% syntactic features at the end of the language<00:15:39.819> that<00:15:39.970> aren't<00:15:40.269> in<00:15:40.420> the<00:15:40.449> kernel 00:15:40.919 --> 00:15:40.929 align:start position:0% language that aren't in the kernel 00:15:40.929 --> 00:15:49.639 align:start position:0% language that aren't in the kernel buffer<00:15:41.139> specification<00:15:42.100> or<00:15:42.300> sentence<00:15:45.569> so<00:15:48.240> I 00:15:49.639 --> 00:15:49.649 align:start position:0% buffer specification or sentence so I 00:15:49.649 --> 00:15:52.530 align:start position:0% buffer specification or sentence so I believe<00:15:50.649> the<00:15:51.040> syntactic<00:15:51.519> issues<00:15:52.089> are<00:15:52.329> not<00:15:52.480> a 00:15:52.530 --> 00:15:52.540 align:start position:0% believe the syntactic issues are not a 00:15:52.540 --> 00:15:58.619 align:start position:0% believe the syntactic issues are not a problem<00:15:52.929> right<00:15:53.139> now<00:15:55.529> so<00:15:56.939> so<00:15:57.939> the<00:15:58.029> thing<00:15:58.059> the<00:15:58.480> me 00:15:58.619 --> 00:15:58.629 align:start position:0% problem right now so so the thing the me 00:15:58.629 --> 00:16:00.840 align:start position:0% problem right now so so the thing the me child<00:15:58.899> is<00:15:59.199> to<00:15:59.439> keep<00:15:59.619> up<00:15:59.800> with<00:16:00.069> the<00:16:00.519> language 00:16:00.840 --> 00:16:00.850 align:start position:0% child is to keep up with the language 00:16:00.850 --> 00:16:03.269 align:start position:0% child is to keep up with the language improvement<00:16:01.360> so<00:16:01.569> a<00:16:01.899> new<00:16:02.439> language<00:16:02.769> features 00:16:03.269 --> 00:16:03.279 align:start position:0% improvement so a new language features 00:16:03.279 --> 00:16:05.489 align:start position:0% improvement so a new language features get<00:16:03.879> added<00:16:04.209> and<00:16:04.420> that<00:16:04.449> has<00:16:04.720> to<00:16:04.839> be<00:16:04.869> reflected 00:16:05.489 --> 00:16:05.499 align:start position:0% get added and that has to be reflected 00:16:05.499 --> 00:16:08.609 align:start position:0% get added and that has to be reflected in<00:16:05.529> the<00:16:05.740> proof<00:16:06.550> of<00:16:06.759> spec<00:16:07.089> so<00:16:07.499> essentially<00:16:08.499> to 00:16:08.609 --> 00:16:08.619 align:start position:0% in the proof of spec so essentially to 00:16:08.619 --> 00:16:11.429 align:start position:0% in the proof of spec so essentially to keep<00:16:08.769> up<00:16:08.980> with<00:16:09.189> it<00:16:09.339> and<00:16:09.660> making<00:16:10.660> sure<00:16:10.809> that 00:16:11.429 --> 00:16:11.439 align:start position:0% keep up with it and making sure that 00:16:11.439 --> 00:16:14.699 align:start position:0% keep up with it and making sure that when<00:16:12.040> there's<00:16:12.249> some<00:16:12.399> change<00:16:12.959> that<00:16:13.959> change<00:16:14.589> is 00:16:14.699 --> 00:16:14.709 align:start position:0% when there's some change that change is 00:16:14.709 --> 00:16:17.629 align:start position:0% when there's some change that change is covered<00:16:15.100> in<00:16:15.339> the<00:16:15.490> closet<00:16:15.790> process<00:16:16.149> so<00:16:16.629> yeah 00:16:17.629 --> 00:16:17.639 align:start position:0% covered in the closet process so yeah 00:16:17.639 --> 00:16:21.809 align:start position:0% covered in the closet process so yeah yeah<00:16:18.639> maybe<00:16:18.819> it<00:16:19.089> was<00:16:19.269> missing<00:16:19.660> is<00:16:19.959> to<00:16:20.619> check<00:16:21.490> if 00:16:21.809 --> 00:16:21.819 align:start position:0% yeah maybe it was missing is to check if 00:16:21.819 --> 00:16:25.139 align:start position:0% yeah maybe it was missing is to check if all<00:16:23.009> broad<00:16:24.009> statements<00:16:24.610> inside<00:16:24.939> the 00:16:25.139 --> 00:16:25.149 align:start position:0% all broad statements inside the 00:16:25.149 --> 00:16:28.139 align:start position:0% all broad statements inside the optimizer<00:16:25.600> steps<00:16:25.959> are<00:16:26.170> powered<00:16:26.879> at<00:16:27.879> a<00:16:27.910> very 00:16:28.139 --> 00:16:28.149 align:start position:0% optimizer steps are powered at a very 00:16:28.149 --> 00:16:29.879 align:start position:0% optimizer steps are powered at a very low<00:16:28.329> level<00:16:28.420> but<00:16:28.899> at<00:16:29.050> the<00:16:29.079> high<00:16:29.290> level<00:16:29.589> it's<00:16:29.709> not 00:16:29.879 --> 00:16:29.889 align:start position:0% low level but at the high level it's not 00:16:29.889 --> 00:16:35.460 align:start position:0% low level but at the high level it's not saying 00:16:35.460 --> 00:16:35.470 align:start position:0% 00:16:35.470 --> 00:16:39.530 align:start position:0% all<00:16:36.470> they<00:16:36.620> do<00:16:36.770> is<00:16:36.950> decide<00:16:37.520> what 00:16:39.530 --> 00:16:39.540 align:start position:0% all they do is decide what 00:16:39.540 --> 00:16:42.420 align:start position:0% all they do is decide what -<00:16:40.540> our<00:16:40.780> problem<00:16:41.110> you<00:16:41.650> are<00:16:41.680> introducing 00:16:42.420 --> 00:16:42.430 align:start position:0% - our problem you are introducing 00:16:42.430 --> 00:16:47.070 align:start position:0% - our problem you are introducing someone<00:16:42.880> else<00:16:43.270> oh<00:16:43.600> they<00:16:43.810> will 00:16:47.070 --> 00:16:47.080 align:start position:0% 00:16:47.080 --> 00:16:52.510 align:start position:0% site<00:16:48.080> is<00:16:48.760> at<00:16:49.760> least<00:16:49.970> four<00:16:50.360> rounds<00:16:51.520> so 00:16:52.510 --> 00:16:52.520 align:start position:0% site is at least four rounds so 00:16:52.520 --> 00:16:55.240 align:start position:0% site is at least four rounds so regarding<00:16:53.120> gas<00:16:53.330> plus<00:16:53.690> the<00:16:54.140> new<00:16:54.590> interpreter 00:16:55.240 --> 00:16:55.250 align:start position:0% regarding gas plus the new interpreter 00:16:55.250 --> 00:16:57.550 align:start position:0% regarding gas plus the new interpreter is<00:16:55.490> oblivious<00:16:56.270> to<00:16:56.300> guess<00:16:56.630> there's<00:16:56.960> no<00:16:57.110> notion 00:16:57.550 --> 00:16:57.560 align:start position:0% is oblivious to guess there's no notion 00:16:57.560 --> 00:17:01.600 align:start position:0% is oblivious to guess there's no notion of<00:16:57.620> gasoline<00:16:58.130> it<00:16:58.520> dis<00:16:58.730> runs<00:16:58.970> good<00:17:00.610> regarding 00:17:01.600 --> 00:17:01.610 align:start position:0% of gasoline it dis runs good regarding 00:17:01.610 --> 00:17:06.730 align:start position:0% of gasoline it dis runs good regarding how<00:17:01.850> we<00:17:01.880> decide<00:17:02.420> basically<00:17:05.470> there's<00:17:06.470> a<00:17:06.530> buck 00:17:06.730 --> 00:17:06.740 align:start position:0% how we decide basically there's a buck 00:17:06.740 --> 00:17:10.660 align:start position:0% how we decide basically there's a buck file<00:17:07.070> by<00:17:07.579> the<00:17:07.640> fuzzing<00:17:08.360> program<00:17:09.260> and<00:17:09.500> we<00:17:10.220> it 00:17:10.660 --> 00:17:10.670 align:start position:0% file by the fuzzing program and we it 00:17:10.670 --> 00:17:13.840 align:start position:0% file by the fuzzing program and we it has<00:17:10.880> minimized<00:17:11.540> input<00:17:12.050> we<00:17:12.829> try<00:17:13.040> to<00:17:13.100> read<00:17:13.670> on 00:17:13.840 --> 00:17:13.850 align:start position:0% has minimized input we try to read on 00:17:13.850 --> 00:17:16.530 align:start position:0% has minimized input we try to read on the<00:17:14.000> input<00:17:14.390> and<00:17:14.570> check<00:17:15.260> the<00:17:15.440> side<00:17:15.620> effects 00:17:16.530 --> 00:17:16.540 align:start position:0% the input and check the side effects 00:17:16.540 --> 00:17:19.660 align:start position:0% the input and check the side effects it's<00:17:17.540> pretty<00:17:17.890> simple<00:17:18.890> workflow<00:17:19.160> maybe 00:17:19.660 --> 00:17:19.670 align:start position:0% it's pretty simple workflow maybe 00:17:19.670 --> 00:17:24.730 align:start position:0% it's pretty simple workflow maybe because<00:17:20.120> of<00:17:20.770> the<00:17:21.940> interpreter<00:17:22.940> so<00:17:23.650> here<00:17:24.650> the 00:17:24.730 --> 00:17:24.740 align:start position:0% because of the interpreter so here the 00:17:24.740 --> 00:17:27.189 align:start position:0% because of the interpreter so here the execution<00:17:25.310> phase<00:17:25.459> which<00:17:25.730> is<00:17:25.760> solid<00:17:26.750> beautiful 00:17:27.189 --> 00:17:27.199 align:start position:0% execution phase which is solid beautiful 00:17:27.199 --> 00:17:29.950 align:start position:0% execution phase which is solid beautiful and<00:17:27.440> you<00:17:28.130> compare<00:17:28.570> execution<00:17:29.570> Croesus 00:17:29.950 --> 00:17:29.960 align:start position:0% and you compare execution Croesus 00:17:29.960 --> 00:17:32.380 align:start position:0% and you compare execution Croesus reinforced<00:17:30.860> and<00:17:31.150> it's<00:17:32.150> pretty 00:17:32.380 --> 00:17:32.390 align:start position:0% reinforced and it's pretty 00:17:32.390 --> 00:17:37.150 align:start position:0% reinforced and it's pretty straightforward<00:17:33.370> so<00:17:34.930> it<00:17:35.930> can<00:17:36.200> quickly<00:17:36.620> tell 00:17:37.150 --> 00:17:37.160 align:start position:0% straightforward so it can quickly tell 00:17:37.160 --> 00:17:39.810 align:start position:0% straightforward so it can quickly tell you<00:17:37.340> whether<00:17:37.730> a<00:17:38.060> bug<00:17:38.300> in<00:17:38.450> the<00:17:38.540> optimizer<00:17:39.020> or 00:17:39.810 --> 00:17:39.820 align:start position:0% you whether a bug in the optimizer or 00:17:39.820 --> 00:17:42.040 align:start position:0% you whether a bug in the optimizer or there's<00:17:40.820> some<00:17:41.000> other<00:17:41.120> poor<00:17:41.360> of<00:17:41.630> course<00:17:41.870> like 00:17:42.040 --> 00:17:42.050 align:start position:0% there's some other poor of course like 00:17:42.050 --> 00:17:43.930 align:start position:0% there's some other poor of course like we<00:17:42.140> said<00:17:42.350> we<00:17:42.980> introduced<00:17:43.340> code<00:17:43.640> and<00:17:43.820> there 00:17:43.930 --> 00:17:43.940 align:start position:0% we said we introduced code and there 00:17:43.940 --> 00:17:45.310 align:start position:0% we said we introduced code and there could<00:17:44.120> be<00:17:44.150> a<00:17:44.270> bug<00:17:44.510> in<00:17:44.660> hood<00:17:44.960> that<00:17:45.110> we 00:17:45.310 --> 00:17:45.320 align:start position:0% could be a bug in hood that we 00:17:45.320 --> 00:17:48.760 align:start position:0% could be a bug in hood that we introduced<00:17:46.120> but<00:17:47.120> it<00:17:48.080> doesn't<00:17:48.380> take<00:17:48.500> longer 00:17:48.760 --> 00:17:48.770 align:start position:0% introduced but it doesn't take longer 00:17:48.770 --> 00:17:51.610 align:start position:0% introduced but it doesn't take longer than<00:17:49.010> a<00:17:50.170> few<00:17:51.170> minutes 00:17:51.610 --> 00:17:51.620 align:start position:0% than a few minutes 00:17:51.620 --> 00:17:54.100 align:start position:0% than a few minutes typically<00:17:52.610> decide<00:17:53.000> whether<00:17:53.240> which<00:17:53.570> bucket<00:17:53.990> it 00:17:54.100 --> 00:17:54.110 align:start position:0% typically decide whether which bucket it 00:17:54.110 --> 00:17:54.930 align:start position:0% typically decide whether which bucket it was<00:17:54.320> it 00:17:54.930 --> 00:17:54.940 align:start position:0% was it 00:17:54.940 --> 00:18:03.630 align:start position:0% was it [Music] 00:18:03.630 --> 00:18:03.640 align:start position:0% 00:18:03.640 --> 00:18:08.770 align:start position:0% [Music] 00:18:08.770 --> 00:18:08.780 align:start position:0% 00:18:08.780 --> 00:18:15.280 align:start position:0% [Applause]