Hugetlbfs support based on empty dir volume plugin #50072

squall0gd · 2017-08-03T08:24:22Z

What this PR does / why we need it: Support for huge pages in empty dir volume plugin. More information about hugepages can be found here

Feature track issue: kubernetes/enhancements#275

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

Release note:

Support for Huge pages in empty_dir volume plugin
[Huge pages](https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt) can now be used with empty dir volume plugin.

k8s-ci-robot · 2017-08-03T08:24:29Z

Hi @squall0gd. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

dixudx · 2017-08-03T08:57:11Z

staging/src/k8s.io/api/core/v1/types.go

@@ -699,6 +699,8 @@ type EmptyDirVolumeSource struct {
 	// More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir
 	// +optional
 	Medium StorageMedium `json:"medium,omitempty" protobuf:"bytes,1,opt,name=medium,casttype=StorageMedium"`
+	// +optional
+	HugetlbfsSize string `json:"hugetlbfsSize,omitempty" protobuf:"bytes,3,opt,name=hugetlbfsSize"`


Since HugetlbfsSize is new-added. You'd need to run auto-gen scripts again (make update will run all auto-gen scripts).

dixudx · 2017-08-03T09:07:25Z

pkg/volume/empty_dir/empty_dir.go

+	if isMnt && medium == mediumHugepages {
+		return nil
+	}
+	if ed.size == "" {


Does this mean that if we are using StorageMediumHugetlbfs, the size is also needed alongside?

If so, you'd better add some validations in pkg/api/validation/validation.go.

It's required in that case. I will move this validation to pkg/api/validation/validation.go

dixudx · 2017-08-03T09:08:57Z

pkg/api/types.go

@@ -614,6 +614,8 @@ type EmptyDirVolumeSource struct {
 	// The default is "" which means to use the node's default medium.
 	// +optional
 	Medium StorageMedium
+	// +optional
+	HugetlbfsSize string


It's kind of confusing to declare xxSize as string. May it be int?

I'll use Quantity here :)

+1 to quantity

dixudx · 2017-08-03T09:09:19Z

pkg/volume/empty_dir/empty_dir.go

@@ -172,6 +179,7 @@ type emptyDir struct {
 	mounter       mount.Interface
 	mountDetector mountDetector
 	plugin        *emptyDirPlugin
+	size          string


Maybe int?

I think it's better to use Quantity here b/c I need to keep unit.

xiangpengzhao · 2017-08-03T11:55:21Z

@squall0gd FYI, there is a proposal on HugePages here: kubernetes/community#837

/cc @derekwaynecarr

derekwaynecarr · 2017-08-09T04:15:34Z

i want to make sure we have agreement on the proposal referenced before merging this, but thank you for your help!

derekwaynecarr · 2017-08-09T04:15:57Z

/ok-to-test

derekwaynecarr · 2017-08-09T07:54:40Z

pkg/api/types.go

-	StorageMediumMemory  StorageMedium = "Memory" // use memory (tmpfs)
+	StorageMediumDefault   StorageMedium = ""          // use whatever the default is for the node
+	StorageMediumMemory    StorageMedium = "Memory"    // use memory (tmpfs)
+	StorageMediumHugetlbfs StorageMedium = "Hugetlbfs" // use hugepages


prefer HugePages

derekwaynecarr · 2017-08-09T07:59:00Z

pkg/volume/empty_dir/empty_dir.go

+	}
+
+	glog.V(3).Infof("pod %v: mounting hugepages for volume %v", ed.pod.UID, ed.volName)
+	mountOptions := []string{fmt.Sprintf("size=%s", ed.size)}


i don't understand why you have size.

size is bounded by sum resource requirement across containers on pod.

we need a new field specific for hugepages that aligns with pagesize. mount option.

well I've used size w/o any reason - just want some, easy to track parameter, to verify that passing mount options is working correctly. Actually I'm waiting for proposal's merge to finalize implementing mount options and to play with unit tests. :)

derekwaynecarr · 2017-09-01T21:51:44Z

/lgtm

Thanks @squall0gd for your help getting this support.

fejta-bot · 2017-09-02T00:57:52Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

PiotrProkop · 2017-09-02T08:36:12Z

/test pull-kubernetes-e2e-kops-aws

squall0gd · 2017-09-02T11:49:45Z

/retest

squall0gd · 2017-09-02T16:11:26Z

/retest

fejta-bot · 2017-09-03T02:09:50Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to @fejta).

Review the full test history for this PR.

eparis · 2017-09-05T15:29:19Z

/lgtm

k8s-github-robot · 2017-09-05T15:29:31Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: childsb, derekwaynecarr, eparis, smarterclayton, squall0gd

Associated issue: 275

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

~~pkg/OWNERS~~ [smarterclayton]
~~plugin/pkg/scheduler/OWNERS~~ [smarterclayton]
~~staging/src/k8s.io/api/core/OWNERS~~ [smarterclayton]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

squall0gd · 2017-09-05T17:46:20Z

/retest

k8s-ci-robot · 2017-09-05T20:01:52Z

@squall0gd: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-kubernetes-cross	`59a86e4`	link	`/test pull-kubernetes-cross`
pull-kubernetes-e2e-kops-aws	`59a86e4`	link	`/test pull-kubernetes-e2e-kops-aws`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-github-robot · 2017-09-05T20:28:37Z

Automatic merge from submit-queue

jdumars · 2017-09-05T21:48:12Z

Curious how this got merged despite failing cross:
I0905 18:14:23.457] pkg/volume/empty_dir/empty_dir_linux.go:54: constant 2508478710 overflows int32
@squall0gd did this error always present in testing and it just got missed?

squall0gd · 2017-09-05T21:55:40Z

@jdumars didn't notice this error before. Just some timeouts on e2e gates

jdumars · 2017-09-05T22:00:59Z

The test will need to be updated in order to accommodate the change.

squall0gd · 2017-09-05T22:12:23Z

@jdumars Well it looks like we need to add type to magic number constant. Sadly we need to do this in another PR.

liggitt · 2017-09-05T22:27:22Z

opened #51984 to type-cast the comparison to the constant

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 3, 2017

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Aug 3, 2017

k8s-github-robot assigned soltysh and eparis Aug 3, 2017

squall0gd force-pushed the squall0gd/hugepages_support branch from cbd37ac to 9f2fcf8 Compare August 3, 2017 08:25

k8s-github-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 3, 2017

dixudx reviewed Aug 3, 2017

View reviewed changes

k8s-ci-robot requested a review from derekwaynecarr August 3, 2017 11:55

k8s-github-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 3, 2017

derekwaynecarr added this to the v1.8 milestone Aug 9, 2017

k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Aug 9, 2017

derekwaynecarr self-assigned this Aug 9, 2017

derekwaynecarr reviewed Aug 9, 2017

View reviewed changes

squall0gd force-pushed the squall0gd/hugepages_support branch from 7148723 to 3c0861b Compare August 16, 2017 10:09

k8s-github-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 16, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 1, 2017

k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 3, 2017

derekwaynecarr and others added 6 commits September 5, 2017 09:46

HugePage changes in API and server

afd8045

Kubelet changes to support hugepages

1ec2a69

Scheduler support for hugepages

41a4e2c

Node validation restricts pre-allocated hugepages to single page size

38d5dee

Support for hugetlbfs in empty dir volume plugin

2dba8f1

Adding getHugePagesMountOptions function and tests

59a86e4

squall0gd force-pushed the squall0gd/hugepages_support branch from 073ac49 to 59a86e4 Compare September 5, 2017 15:20

k8s-github-robot removed lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Sep 5, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 5, 2017

k8s-github-robot merged commit fa191ed into kubernetes:master Sep 5, 2017

cblecker mentioned this pull request Sep 5, 2017

cross-build: PR 50072 causing cross build failure #51978

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hugetlbfs support based on empty dir volume plugin #50072

Hugetlbfs support based on empty dir volume plugin #50072

squall0gd commented Aug 3, 2017 •

edited by derekwaynecarr

Loading

k8s-ci-robot commented Aug 3, 2017

dixudx Aug 3, 2017

squall0gd Aug 3, 2017

dixudx Aug 3, 2017

squall0gd Aug 3, 2017

dixudx Aug 3, 2017

squall0gd Aug 3, 2017

derekwaynecarr Aug 9, 2017

dixudx Aug 3, 2017 •

edited

Loading

squall0gd Aug 3, 2017

xiangpengzhao commented Aug 3, 2017

derekwaynecarr commented Aug 9, 2017

derekwaynecarr commented Aug 9, 2017

derekwaynecarr Aug 9, 2017

derekwaynecarr Aug 9, 2017

squall0gd Aug 9, 2017

derekwaynecarr commented Sep 1, 2017

fejta-bot commented Sep 2, 2017

PiotrProkop commented Sep 2, 2017

squall0gd commented Sep 2, 2017

squall0gd commented Sep 2, 2017

fejta-bot commented Sep 3, 2017

eparis commented Sep 5, 2017

k8s-github-robot commented Sep 5, 2017

squall0gd commented Sep 5, 2017

k8s-ci-robot commented Sep 5, 2017 •

edited

Loading

k8s-github-robot commented Sep 5, 2017

jdumars commented Sep 5, 2017

squall0gd commented Sep 5, 2017

jdumars commented Sep 5, 2017

squall0gd commented Sep 5, 2017

liggitt commented Sep 5, 2017

Hugetlbfs support based on empty dir volume plugin #50072

Hugetlbfs support based on empty dir volume plugin #50072

Conversation

squall0gd commented Aug 3, 2017 • edited by derekwaynecarr Loading

k8s-ci-robot commented Aug 3, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dixudx Aug 3, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xiangpengzhao commented Aug 3, 2017

derekwaynecarr commented Aug 9, 2017

derekwaynecarr commented Aug 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

derekwaynecarr commented Sep 1, 2017

fejta-bot commented Sep 2, 2017

PiotrProkop commented Sep 2, 2017

squall0gd commented Sep 2, 2017

squall0gd commented Sep 2, 2017

fejta-bot commented Sep 3, 2017

eparis commented Sep 5, 2017

k8s-github-robot commented Sep 5, 2017

squall0gd commented Sep 5, 2017

k8s-ci-robot commented Sep 5, 2017 • edited Loading

k8s-github-robot commented Sep 5, 2017

jdumars commented Sep 5, 2017

squall0gd commented Sep 5, 2017

jdumars commented Sep 5, 2017

squall0gd commented Sep 5, 2017

liggitt commented Sep 5, 2017

squall0gd commented Aug 3, 2017 •

edited by derekwaynecarr

Loading

dixudx Aug 3, 2017 •

edited

Loading

k8s-ci-robot commented Sep 5, 2017 •

edited

Loading