[Telemetry] Use --ignore-certificate-errors-spki-list to bypass cert errors

[Telemetry] Use --ignore-certificate-errors-spki-list to bypass cert errors

--ignore-certificate-errors-spki-list is a new flag that can be used to
selectively bypass certificate errors. This flag avoid the downsides of
the old --ignore-certificate-errors flag (which makes Chrome to skip 
disk cache and re-establish socket connections) For more details, please 
see the linked bug below.

BUG=chromium:753948
Review-Url: https://chromiumcodereview.appspot.com/3003143002
Committed: https://chromium.googlesource.com/external/github.com/catapult-project/catapult/+/724bb776bcdea642a5bc9cc20fe14bb0282b4900

[xunjieli, 2017-08-22 15:12:00]

The CQ bit was checked by xunjieli@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-22 15:12:04]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-08-22 15:15:14]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-22 15:15:15]

Dry run: Try jobs failed on following builders:
  Catapult Presubmit on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Pr...)

[xunjieli, 2017-08-22 15:16:21]

xunjieli@chromium.org changed reviewers:
+ nednguyen@google.com

[xunjieli, 2017-08-22 15:16:21]

Ned, PTAL? 
The old Python WPR and the new Golang WPR use the same fake root, so the key
hash should be the same.

I tested this locally. System health benchmarks are passing.

[xunjieli, 2017-08-22 15:17:10]

Description was changed from

==========
[Telemetry] Use --ignore-certificate-errors-spki-list to bypass cert errors

--ignore-certificate-errors-spki-list is a new flag that can be used to
selectively bypass certificate errors. This flag avoid the downsides of
the old --ignore-certificate-errors flag. For more details, please see the
linked bug below.

BUG=chromium:#753948
==========

to

==========
[Telemetry] Use --ignore-certificate-errors-spki-list to bypass cert errors

--ignore-certificate-errors-spki-list is a new flag that can be used to
selectively bypass certificate errors. This flag avoid the downsides of
the old --ignore-certificate-errors flag. For more details, please see the
linked bug below.

BUG=chromium:753948
==========

[xunjieli, 2017-08-22 15:24:38]

The CQ bit was checked by xunjieli@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-22 15:24:44]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[nednguyen, 2017-08-22 15:40:50]

I am excited by this change!

https://codereview.chromium.org/3003143002/diff/20001/telemetry/telemetry/int...
File telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py
(right):

https://codereview.chromium.org/3003143002/diff/20001/telemetry/telemetry/int...
telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py:135: #
base64
I think this string probably should be managed by wprgo code base. Maybe stored
in a file like ca_root.txt.

Then with this, we can add PRESUBMIT so that if people change wpr_cert.pem,
presubmit will validate whether the string in ca_root.txt is in sync.

https://codereview.chromium.org/3003143002/diff/20001/telemetry/telemetry/int...
telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py:136:
replay_args.append('--ignore-certificate-errors-spki-list='
nits: also link the discussion in crbug.com/757508 to here

[commit-bot: I haz the power, 2017-08-22 15:51:26]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-22 15:51:27]

Dry run: Try jobs failed on following builders:
  Catapult Android Tryserver on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20An...)

[xunjieli, 2017-08-22 21:05:00]

Patchset #3 (id:40001) has been deleted

[xunjieli, 2017-08-22 21:05:43]

Thanks, PTAL.

https://codereview.chromium.org/3003143002/diff/20001/telemetry/telemetry/int...
File telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py
(right):

https://codereview.chromium.org/3003143002/diff/20001/telemetry/telemetry/int...
telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py:135: #
base64
On 2017/08/22 15:40:50, nednguyen wrote:
> I think this string probably should be managed by wprgo code base. Maybe
stored
> in a file like ca_root.txt.
> 
> Then with this, we can add PRESUBMIT so that if people change wpr_cert.pem,
> presubmit will validate whether the string in ca_root.txt is in sync. 

Done.

[xunjieli, 2017-08-22 21:05:47]

The CQ bit was checked by xunjieli@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-22 21:06:02]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-08-22 21:09:06]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-22 21:09:07]

Dry run: Try jobs failed on following builders:
  Catapult Presubmit on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Pr...)

[nednguyen, 2017-08-22 21:12:18]

lgtm

https://codereview.chromium.org/3003143002/diff/80001/telemetry/telemetry/int...
File telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py
(right):

https://codereview.chromium.org/3003143002/diff/80001/telemetry/telemetry/int...
telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py:136:
'web_page_replay_go/wpr_public_hash.txt')
nits: 
os.path.join(util.GetCatapultDir(), 'web_page_replay_go', 'wpr_public_hash.txt')

[xunjieli, 2017-08-22 21:28:41]

Thanks, I will file a catapult issue to add the presubmit check.

https://codereview.chromium.org/3003143002/diff/80001/telemetry/telemetry/int...
File telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py
(right):

https://codereview.chromium.org/3003143002/diff/80001/telemetry/telemetry/int...
telemetry/telemetry/internal/backends/chrome/chrome_browser_backend.py:136:
'web_page_replay_go/wpr_public_hash.txt')
On 2017/08/22 21:12:18, nednguyen wrote:
> nits: 
> os.path.join(util.GetCatapultDir(), 'web_page_replay_go',
'wpr_public_hash.txt')

Done.

[xunjieli, 2017-08-22 21:28:45]

The CQ bit was checked by xunjieli@chromium.org

[xunjieli, 2017-08-22 21:28:45]

The patchset sent to the CQ was uploaded after l-g-t-m from nednguyen@google.com
Link to the patchset: https://codereview.chromium.org/3003143002/#ps100001
(title: "address comments")

[commit-bot: I haz the power, 2017-08-22 21:28:47]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-08-22 21:45:37]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-22 21:45:38]

Try jobs failed on following builders:
  Catapult Mac Tryserver on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Ma...)

[xunjieli, 2017-08-22 21:51:04]

The CQ bit was checked by xunjieli@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-22 21:51:08]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-08-22 22:13:57]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-22 22:13:58]

Dry run: Try jobs failed on following builders:
  Catapult Windows Tryserver on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Wi...)

[xunjieli, 2017-08-24 00:20:54]

The CQ bit was checked by xunjieli@chromium.org

[xunjieli, 2017-08-24 00:20:55]

The patchset sent to the CQ was uploaded after l-g-t-m from nednguyen@google.com
Link to the patchset: https://codereview.chromium.org/3003143002/#ps120001
(title: "strip newline")

[commit-bot: I haz the power, 2017-08-24 00:21:01]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-08-24 01:00:30]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-24 01:00:31]

Try jobs failed on following builders:
  Catapult Android Tryserver on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20An...)

[xunjieli, 2017-08-24 13:26:32]

The CQ bit was checked by xunjieli@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-24 13:26:40]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-08-24 14:07:10]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-24 14:07:11]

Dry run: Try jobs failed on following builders:
  Catapult Android Tryserver on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20An...)

[xunjieli, 2017-08-24 18:46:52]

The CQ bit was checked by xunjieli@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-24 18:47:01]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[xunjieli, 2017-08-24 18:49:04]

The CQ bit was checked by xunjieli@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-24 18:49:10]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[xunjieli, 2017-08-24 19:02:07]

Patchset #7 (id:140001) has been deleted

[commit-bot: I haz the power, 2017-08-24 19:19:40]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-24 19:19:41]

Dry run: Try jobs failed on following builders:
  Catapult Linux Tryserver on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Li...)

[xunjieli, 2017-08-24 19:20:42]

Patchset #7 (id:160001) has been deleted

[xunjieli, 2017-08-24 19:25:01]

The CQ bit was checked by xunjieli@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-24 19:25:11]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-08-24 20:55:26]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-24 20:55:27]

Dry run: This issue passed the CQ dry run.

[nednguyen(REVIEW IN OTHER ACC), 2017-08-25 13:57:30]

On 2017/08/24 01:00:31, commit-bot: I haz the power wrote:
> Try jobs failed on following builders:
>   Catapult Android Tryserver on master.tryserver.client.catapult (JOB_FAILED,
>
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20An...)

To enable user data dir for Android, you can email perezju@ for his opinion.
Once he approved, send and email to telemetry-announce@

[xunjieli, 2017-08-30 21:50:57]

The CQ bit was checked by xunjieli@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-30 21:51:07]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-08-30 22:17:57]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-30 22:17:58]

Dry run: This issue passed the CQ dry run.

[xunjieli, 2017-08-31 18:40:02]

The CQ bit was checked by xunjieli@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-08-31 18:40:05]

Dry run: CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[xunjieli, 2017-08-31 18:50:19]

Description was changed from

==========
[Telemetry] Use --ignore-certificate-errors-spki-list to bypass cert errors

--ignore-certificate-errors-spki-list is a new flag that can be used to
selectively bypass certificate errors. This flag avoid the downsides of
the old --ignore-certificate-errors flag. For more details, please see the
linked bug below.

BUG=chromium:753948
==========

to

==========
[Telemetry] Use --ignore-certificate-errors-spki-list to bypass cert errors

--ignore-certificate-errors-spki-list is a new flag that can be used to
selectively bypass certificate errors. This flag avoid the downsides of
the old --ignore-certificate-errors flag (which makes Chrome to skip 
disk cache and re-establish socket connections) For more details, please 
see the linked bug below.

BUG=chromium:753948
==========

[commit-bot: I haz the power, 2017-08-31 19:28:56]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-08-31 19:28:57]

Dry run: This issue passed the CQ dry run.

[xunjieli, 2017-09-01 13:52:52]

All dependent CLs have been landed. I kicked off a few perf trybots. 
If everything goes well, I will land this after labor day weekend.

[xunjieli, 2017-09-05 12:41:22]

xunjieli@chromium.org changed reviewers:
+ simonhatch@chromium.org

[xunjieli, 2017-09-05 12:47:58]

On 2017/09/01 13:52:52, xunjieli wrote:
> All dependent CLs have been landed. I kicked off a few perf trybots. 
> If everything goes well, I will land this after labor day weekend.

+simonhatch@:
Simon, I ran "tools/perf/run_benchmark try all system_health.common_desktop
--repo_path=third_party/catapult".
I can't find "HTLM results" on the build output pages.  For example,
linux_perf_bisect bot
(https://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bise...).
Do you know what I am missing? Thanks!

[xunjieli, 2017-09-06 14:01:27]

On 2017/09/05 12:47:58, xunjieli wrote:
> On 2017/09/01 13:52:52, xunjieli wrote:
> > All dependent CLs have been landed. I kicked off a few perf trybots. 
> > If everything goes well, I will land this after labor day weekend.
> 
> +simonhatch@:
> Simon, I ran "tools/perf/run_benchmark try all system_health.common_desktop
> --repo_path=third_party/catapult".
> I can't find "HTLM results" on the build output pages.  For example,
> linux_perf_bisect bot
>
(https://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bise...).
> Do you know what I am missing? Thanks!

Talked to Simon offline. There has been some change on Telemetry side, and we
need to add --output-format-html to the command.
I re-ran the trybots last night. "timeToFirstContentfulPaint" (which regressed
with WprGo migration) is lower, as expected.

Two sample runs:
https://build.chromium.org/p/tryserver.chromium.perf/builders/mac_retina_perf...
https://build.chromium.org/p/tryserver.chromium.perf/builders/linux_perf_bise...

I am CQ-ing the CL now.

[xunjieli, 2017-09-06 14:02:24]

The CQ bit was checked by xunjieli@chromium.org

[xunjieli, 2017-09-06 14:02:24]

The patchset sent to the CQ was uploaded after l-g-t-m from nednguyen@google.com
Link to the patchset: https://codereview.chromium.org/3003143002/#ps220001
(title: "Rebased")

[commit-bot: I haz the power, 2017-09-06 14:02:27]

CQ is trying da patch.

Follow status at:
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-09-06 14:29:20]

CQ is committing da patch.

Bot data: {"patchset_id": 220001, "attempt_start_ts": 1504706544351670,
"parent_rev": "c5a474f50f94f4df9cf979c1e7accc5515c79ca7", "commit_rev":
"724bb776bcdea642a5bc9cc20fe14bb0282b4900"}

[commit-bot: I haz the power, 2017-09-06 14:29:25]

Description was changed from

==========
[Telemetry] Use --ignore-certificate-errors-spki-list to bypass cert errors

--ignore-certificate-errors-spki-list is a new flag that can be used to
selectively bypass certificate errors. This flag avoid the downsides of
the old --ignore-certificate-errors flag (which makes Chrome to skip 
disk cache and re-establish socket connections) For more details, please 
see the linked bug below.

BUG=chromium:753948
==========

to

==========
[Telemetry] Use --ignore-certificate-errors-spki-list to bypass cert errors

--ignore-certificate-errors-spki-list is a new flag that can be used to
selectively bypass certificate errors. This flag avoid the downsides of
the old --ignore-certificate-errors flag (which makes Chrome to skip 
disk cache and re-establish socket connections) For more details, please 
see the linked bug below.

BUG=chromium:753948

Review-Url: https://chromiumcodereview.appspot.com/3003143002
Committed:
https://chromium.googlesource.com/external/github.com/catapult-project/catapu...
==========

[commit-bot: I haz the power, 2017-09-06 14:29:26]

Committed patchset #9 (id:220001) as
https://chromium.googlesource.com/external/github.com/catapult-project/catapu...

[perezju, 2017-09-11 15:12:13]

A revert of this CL (patchset #9 id:220001) has been created in
https://chromiumcodereview.appspot.com/3009313002/ by perezju@chromium.org.

The reason for reverting is: Broke perf tests for webview crbug.com/763880.