Golang - Windows and Linux Log collection


We have a REST API that’s used to receive log data and provide configurations for what information to log from a host. We would like to develop a Golang application to retrieve logs and basic metrics from a host. This application will run on a client and send logs to the REST API. The client side application is not responsible for parsing logs (no regex needed) just collecting the information and sending it. This will be installed on Windows or Linux as a service and would need to be constantly online.

The functionality is similar to the beats products such as Filebeats, Winlogbeats, and Metricbeats. As these are opensource and written in Go, it is fine to use parts of the applications but we’re not interested in just making a plugin for these to adapt to our system. Unlike the beats products that have many options, we want to focus on just the key functions below.

Retrieve configuration from API:

By doing a REST get and providing the host id, the API will return a configuration that defines which logs to retrieve and which metrics to collect.

Collecting logs from Windows:

The application should query Windows event logs (System, Application, and Security), convert to json format, and then forward to our REST API. The application should keep track so that it does not read in duplicate logs.

Collecting logs from Linux and files in Windows:

The logs are defined in the configuration with a wildcard to support log rotate. Unlike systems like Splunk or beats, we do not need to monitor all files in a directory. We really only need to support monitoring a single file, but support collecting from a rotated file when the log is rotated. Our current method is to monitor a single file, but when a file is smaller than it was when last read, we look to the rotated file to ensure no logs were missed.

Metrics from Linux and Windows:

Though other system collect detailed metrics, we support fairly basic collection of metrics. We do plan to expand these in the future but for now we are just collecting these items.

-Processes with RAM and CPU

-Services with current state


-RAM utilization

-CPU utilization

-Disk utilization

Host Information:

We collect basic information about the host infrequently. This is typically done on an hourly or daily basis.



-Processor info

-Ram info

-Open ports

-List Installed applications

-Network information (Interfaces, IPs, etc…)

Kemahiran: Golang, Linux, Kejuruteraan Perisian

Lihat lagi: golang log to file and stdout, golang log to file, golang log rotation, golang logging best practices, golang centralized logging, golang log server, golang google log, golang read windows event log, Windows Linux, windows services log file, windows 2003 log logins, voipswitch platform windows linux, program windows linux, windows service log files, windows service log, windows linux networking, linux log automated, linux log parser insert mysql, transfer code windows linux, windows linux php generate thumbnail web page

Tentang Majikan:
( 16 ulasan ) Anchorage, United States

ID Projek: #19233753

Dianugerahkan kepada:


Hey there, I can build you what you're looking for. I have over six years of software development experience and ten years of system administration. Similar experience I have is building remote clients for a cont Lagi

$750 USD dalam 14 hari
(0 Ulasan)

14 pekerja bebas membida secara purata $830 untuk pekerjaan ini


Hi, I can make a rest api service for you to pick up logs that you requested windows/linux os's. If interested, contact me.

$750 USD dalam 7 hari
(13 Ulasan)

Hello, let me help you with your Golang/Python/REST/subprocess/websockets project. I have good experience also in Kubernetes and Docker with UI on Flask/Python. I have deep knowledge in CI/CD software development as on Lagi

$777 USD dalam 10 hari
(32 Ulasan)

Hi, sir Thank you for reading my bid. I have already completed a lot of back end by using Go lang. Especially, I have rich experience micro service back end development. As you know, Go lang is very excellent in pe Lagi

$666 USD dalam 10 hari
(6 Ulasan)

Hi. hope you are doing well. I'm a golang developer which have 12 years of development experience on windows and linux. No problem to implement your require. I'm looking forward to hearing from you. Thanks, regards Lagi

$1000 USD dalam 10 hari
(1 Ulasan)

Hello, my name is Oliver, I work with golang for over 5 years. Based on the specifications I read in your project, this repository will probably help with most of the data that needs to be collected: https://github Lagi

$440 USD dalam 10 hari
(3 Ulasan)

We already did the log collection project for [login to view URL] check our previous [login to view URL] already did this for REST [login to view URL] we can easily do this for you

$1333 USD dalam 21 hari
(1 Ulasan)

Hi! How are you? I read your project description and I am really interested in your project. I am a senior software developer who has many experience with windows and linux OS. I am sure that i could implement your lo Lagi

$500 USD dalam 10 hari
(2 Ulasan)

I have done a golang project named log collector, this project parses rotated log files line by line and send it via rest api, your requirement is just upload the whole file via rest or or just send line by line and ad Lagi

$250 USD dalam 10 hari
(3 Ulasan)

We are a team of experienced full stack developer professionals who will love to help you in achieving your goals. We assure you to deliver the best quality work and within the desired time. We will complete your pro Lagi

$633 USD dalam 40 hari
(0 Ulasan)

Will give you on time service.

$444 USD dalam 10 hari
(0 Ulasan)

Meeting spec requirements + CI pipeline/installation package creation (deb/RPM/exe) and unit tests.

$2777 USD dalam 7 hari
(0 Ulasan)

I have done golang projects for windows, macos, linux and raspbian. Most golang projects i have done will need to retrieve the system logs or parameters.

$750 USD dalam 30 hari
(0 Ulasan)

Recently I have worked with system monitoring and log collection for a project. We used EFK stack and NodeJS. I think it would be easier to monitor the system logs. We just start a go-routine and wait for logs changes Lagi

$555 USD dalam 10 hari
(0 Ulasan)