Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added support for tail to query frontend #2032

Merged
merged 9 commits into from
Jul 24, 2020
Merged

Added support for tail to query frontend #2032

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
d02f8c7
Added support for tail to query frontend
tivvit May 2, 2020
a12473f
cleanup WsTailProxy
tivvit May 2, 2020
418b5b2
merge master
tivvit Jul 21, 2020
0a209d4
used httputil.NewSingleHostReverseProxy insted of custom implementati…
tivvit Jul 21, 2020
e690786
proxy all unknown requests
tivvit Jul 21, 2020
1a87f60
rename proxy to tailProxy
tivvit Jul 23, 2020
ef61f93
default handler, if tailproxy url is set up default proxy is tailproxy
tivvit Jul 23, 2020
bb23ee3
embedded lokifrontend conf which adds tail_proxy_url
tivvit Jul 23, 2020
8ce9891
fixed tail_proxy_url flag
tivvit Jul 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions pkg/loki/loki.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ import (
"github.com/grafana/loki/pkg/querier"
"github.com/grafana/loki/pkg/querier/queryrange"
"github.com/grafana/loki/pkg/storage"
"github.com/grafana/loki/pkg/tailproxy"
"github.com/grafana/loki/pkg/tracing"
serverutil "github.com/grafana/loki/pkg/util/server"
"github.com/grafana/loki/pkg/util/validation"
Expand All @@ -55,6 +56,7 @@ type Config struct {
TableManager chunk.TableManagerConfig `yaml:"table_manager,omitempty"`
Worker frontend.WorkerConfig `yaml:"frontend_worker,omitempty"`
Frontend frontend.Config `yaml:"frontend,omitempty"`
TailProxy tailproxy.Config `yaml:"tail_proxy,omitempty"`
Copy link
Member

@owen-d owen-d Jul 23, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to what @sandeepsukhani mentioned, I think this should be part of the frontend config rather than it's own top level value because it's only used in the frontend.

We may need to extend our frontend config definition to be something like

type Config struct {
    frontend.Config `yaml:",inline"`
    tail_proxy_url string `yaml:"tail_proxy_url"`
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good idea. I did it like this because I knew it does not belong to Cortex.

This is better I will prepare it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

edit: i love the podracer gif :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@owen-d what do think about it now? I used embedded struct as you proposed and IMHO it is better.

QueryRange queryrange.Config `yaml:"query_range,omitempty"`
RuntimeConfig runtimeconfig.ManagerConfig `yaml:"runtime_config,omitempty"`
MemberlistKV memberlist.KVConfig `yaml:"memberlist"`
Expand Down Expand Up @@ -85,6 +87,7 @@ func (c *Config) RegisterFlags(f *flag.FlagSet) {
c.RuntimeConfig.RegisterFlags(f)
c.MemberlistKV.RegisterFlags(f, "")
c.Tracing.RegisterFlags(f)
c.TailProxy.RegisterFlags(f)
}

// Validate the config and returns an error if the validation
Expand Down
14 changes: 13 additions & 1 deletion pkg/loki/modules.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ import (
"errors"
"fmt"
"net/http"
"net/http/httputil"
"net/url"
"os"
"sort"
"time"
Expand Down Expand Up @@ -297,6 +299,16 @@ func (t *Loki) initQueryFrontend() (_ services.Service, err error) {
serverutil.NewPrepopulateMiddleware(),
).Wrap(t.frontend.Handler())

httpMiddleware := middleware.Merge(
t.httpAuthMiddleware,
queryrange.StatsHTTPMiddleware,
)
tailURL, err := url.Parse(t.cfg.TailProxy.DownstreamURL)
if err != nil {
return
}
tp := httputil.NewSingleHostReverseProxy(tailURL)
proxy := httpMiddleware.Wrap(tp)
tivvit marked this conversation as resolved.
Show resolved Hide resolved
t.server.HTTP.Handle("/loki/api/v1/query_range", frontendHandler)
t.server.HTTP.Handle("/loki/api/v1/query", frontendHandler)
t.server.HTTP.Handle("/loki/api/v1/label", frontendHandler)
Expand All @@ -308,7 +320,7 @@ func (t *Loki) initQueryFrontend() (_ services.Service, err error) {
t.server.HTTP.Handle("/api/prom/label/{name}/values", frontendHandler)
t.server.HTTP.Handle("/api/prom/series", frontendHandler)
// fallback route
t.server.HTTP.PathPrefix("/").Handler(frontendHandler)
t.server.HTTP.PathPrefix("/").Handler(proxy)

return services.NewIdleService(nil, func(_ error) error {
t.frontend.Close()
Expand Down
15 changes: 15 additions & 0 deletions pkg/tailproxy/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
package tailproxy

import (
"flag"
)

// Config for a tailproxy
type Config struct {
DownstreamURL string `yaml:"downstream_url"`
}

// RegisterFlags adds the flags required to config this to the given FlagSet.
func (cfg *Config) RegisterFlags(f *flag.FlagSet) {
f.StringVar(&cfg.DownstreamURL, "tailproxy.downstream-url", "", "URL of downstream querier.")
}