FPGA Mining Guide 08/2018 Thomas Block https:// ht tps://twitter.com/ThomasBlock3 twitter.com/ThomasBlock3 https://www.youtube.com/channel/UCCTIyxOWqhwB3cu7PbRIgQA https://ww w.youtube.com/channel/UCCTIyxOWqhwB3cu7PbRIgQA
Google Sheet Version (for Comments): https://docs.google.com/document/d/1LamZtUdye18UJ_5vewdd38UBdk1c7LC_dsu5ZHdC2 gg/edit?usp=sharing
Xilinx DK-U1-VCU1525-P-G DK-U1-VCU1525-P-G Xilinx DK-U1-VCU1525-P-G DK-U1-VCU1525-P-G Introduction VCU1525 VC U1525 specs How to get it RAM Voucher Vou cher for software design toolkit DC1613 A what? why? how? Hardware Setup Cooling Hardware Connections PCIE Riser Power PCIE Riser USB to mainboard Mini-USB from card from card to mainboard 8 port PCIE power connection LT powerplay device Second Pc Board LEDs Using Vivado Labs Using LT Powerplay Software Mining 0xBTC clocks Mining Tribus Mining Keccak Profit Estimation Real World income Next Steps
Immersion Cooling Theory Coolant Heat Removal my immersion cooling setup Real World income next step
Introduction Hi. In this guide i am a m constantly collecting all the information i learne d about my FPGA cryptocurrency mining experience in 2018. It may be useful for people who want to get into fpga mining. read, learn, share your knowledge. if you think that something is unclear or missing, just contact me and i will update it.
Donations go here (btc): bc1q7lryh7h4qnpgtjjq0pz70wwnkyuaj5lg2mlml7 3DbZzhr6mc2NWZuEY6Fic51SYZ8yqkn48Z
Thomas Block
This Work is based on the hard work of the FPGA developer scene. I appreciate their vision of sharing bitstreams very much. Thank you! you ! Here are some links to their original posts: whitefire990 https://bitcointalk.org/index.php?topic=3459858.0 http://zetheron.com/ senseless https://bitcointalk.org/index.php?topic=3688965.0 https://fpga.land/ GPUHoarder https://bitcointalk.org/index.php?topic=4391318.0 http://www.squirrelsresearch.com/ DigitalCruncher https://bitcointalk.org/index.php?action=profile;u=1735391 https://github.com/digitalcruncher and many more, join us on: https://discord.gg/25DpDCS there are also some other services: https://store.mineority.io https://fpgatalk.org/
https://fpga.guide/en/ https://nocroom.com/fpga
VCU1525 specs https://www.xilinx.com/products/boards-and-kits/vcu1525-a.html manual: https://japan.xilinx.com/support/documentation/boards_and_kits/vcu1525/ug1268-vcu1525-r econfig-accel-platform.pdf
How to get it once listed directly on https://www.avnet.com/shop/emea/ https://www.avnet.com/shop/emea/ now now redireted to EBV Elektronik: https://www.avnet.com/wps/portal/ebv/products/new-products/npi/2018/xilinx-vcu1525-accel eration-kit as of August 2018, you can order it from AVNet USA directly. They get it from Asia, so lead time is 4-8 weeks. They can then express ship it worldwide ( except Cuba, Iran, North Korea, Sudan and Syria ). Price is 4995 USD plus local taxes. Worked well for me in germany. There also was a group buy of 5000 units on https://fpga.land/ https://fpga.land/ - but it is sold out and may also be delayed. so you might be faster by ordering directly. the downside is that you only get that passive cooled version which needs a lot of work for proper cooling. the group variant called BCU1525 has already better cooling installed. ALTERNATIVES 10/2018: https://talentpros.com.tw/%E6%9C%8D%E5%8B%99%E8%88%87%E7%94%A2%E5%93 %81%E4%BB%8B%E7%B4%B9 https://shop.fpga.guide/collections/all/products/btu9p-by-tul
RAM the card comes with 4 * 16 GB RAM, but actual bitstreams don’t need it. so you can remove and even sell it. ( but maybe future software will take profit from / need RAM )
Voucher for software design toolkit it is included in the box. you don’t need it for mining. it says it is valid one year. so maybe you can sell it / resell it when selling the card.
DC1613A
what? http://www.analog.com/media/en/dsp-documentation/evaluation-kit-manuals/dc1613af.pdf this additional LT powerplay device can undervolt the card. there are two modes of operation: ● buy one unit, unit, and and under undervol voltt each each card card to 0.72 0.72 V ● buy one unit per per fpga and control control the the voltages voltages live live while while mining mining ( you can go even lower, but it can crash )
why? the stock value of 0.85V causes a really high power usage - i could not even use the slowest bitstream. with this device i can go to much higher speed, at lower temperatures. see my measurements of power consumption at the wall:
how? you have to buy the device and build your own connector, as described here: http://zetheron.com/Downloads/VCU1525_DC1613A_Adapter.pdf The Software usage is described below.
Hardware Setup
my advice is to use PCIE risers, known from GPU mining: so you can locate the card on a place where you can cool it easily ( with fans e.g. ). and you can be sure you don’t draw too much power from the mainboard. choose a cpu with >0,5 cores per fpga. choose a mainboard with 1 pcie slot per fpga. ( 1x is fine ) choose a quality PSU ( 400 Watt per FPGA, each needs an 8 port pcie power and a riser power ) 4 GB RAM is ok use a ssd. create a dual boot system with windows and linux - so you can run software for either operating system.
Cooling the passive cooled version i got needs a lot of cooling. i experimented with HT water pipes. they are cheap and made for up to 95°C. connected with two antminer fans you get a lot of airflow: ( the mining bazooka )
But i also used industrial fans and pvc air pipes:
Antminer Cooling:
Hardware Connections to get the best results, the card needs FIVE connections.
PCIE Riser Power from psu to riser. the card draws 75W power from here. ( maybe even more? see below )
PCIE Riser USB to mainboard some bitstreams communicate over this bus with the card. ( DigitalCruncher )
Mini-USB from card to mainboard Vivado needs this connection to program the bitstream. some bitstreams communicate over this bus with the card. ( Whitefire ).
8 port PCIE power connection from psu to card. PCIE specs allow 150 Watt. but this card can draw 300W. so don’t use a cheap psu.
LT powerplay device connect your card to the LT device. ( but the usb connection from LT goes to a second pc )
Second Pc you will need a very basic windows pc with usb ( old laptop ). we will connect it to the LT device - so we can control the telemetry of the cards regardless in which state / operating system the first pc is. and usb ports won’t interfere.
Board LEDs
the board has 5 LEDs. my actual guess of assignment: ● blue light light at the the top glowin glowing g = bitst bitstrea ream m is progr programm ammed ed ● red light light below below glowing: glowing: card card is in error error state state -> check LT Powerplay Powerplay,, reset reset ● thre three e lilights ghts in a row row ○ gree green n bli blink nkin ing g = work workin ing g ○ yell yellow ow glowi glowing ng = pcie pcie pow power er is is pres presen entt ○ red: red: ( not not seen seen yet yet )
Using Vivado Labs ● ● ●
registe registerr for free, free, then then you can downl download oad the the free “viva “vivado do lab editi edition” on” https://www.xilinx.com/support/download.html you you can can use use Wind Window ows s or or lin linux ux
if everything works well you can see your boards. if not, the usb connection might be missing, or the board could be in error state (“[Labtools 27-2269] No devices detected on target localhost:3121/xilinx_tcf/Xilinx/12809621t006A. Check cable connectivity and that the target board is powered up then use the disconnect_hw_server and connect_hw_server to re-register this hardware target.“)
Using LT Powerplay Software also check zetherons manual: http://zetheron.com/Downloads/VCU1525_DC1613A_Adapter.pdf install: ● .NET 3.5 https://www.microsoft.com/de-de/download/details.aspx?id=21 ● USB USB port dri driver http://www.ftdichip.com/Drivers/D2XX.htm ● LT Power Play http://www.ltpowerplay.com/download/ open the LT power play. if it is requesting a licence, communication with the card does not work. click “detect chips” problem: you can not use several LT devices on one pc. the data of all connected devices will mix up. so we will only connect one at the time. but you can keep the software running: just change usb cables and it will update. If you start the software and see this, you usb connection to the box is not correct:
you should see this and click “detect chips”
working software could look like this ( i closed some subwindows ):
now you can change the voltage and save it:
Sometimes it happens that you want to save something and error “registers were not written” occurs. Just press 3 times OK and try it again:
if the card is in error state, you have to click the clear button and maybe reset.
Mining 0xBTC you have several fundamental options: ●
●
●
●
mine in a pool ○ easy setup ○ high fees mine solo ○ no fees ○ you need a ETH ETH priva private te key key with with balanc balance e on it mine 0x 0xBTC ○ high high long long term term stab stabililit ity y ○ high high difficu difficulty lty:: solo solo mining mining can can take take up to 10 days days for for one block block mine 0x Al Altcoins ○ low diffic difficult ulty: y: solo solo-mi -mine ne one one Bloc Block k per per hour hour ○ most of them them don’t don’t even even have have an exchange exchange listing listing - could could be completely completely worthless
you need Zetherons Miner, read his great instructions http://zetheron.com/Downloads/VCU1525_Instructions.pdf http://zetheron.com/index.php/downloads/ ● ● ●
http://www.ftdichip.com/Drivers/CDM/CDM21228_Setup.zip https://www.microsoft.com/en-ca/download/details.aspx?id=40784 unzi unzip p mine minerr soft softwa ware re
for pool mining ● edit edit tok token enmi mine ner. r.iini ○ Miner MinerAc Acct ct=0 =0x0 x0000 0000y 0you ourp rpubl ublic icke key y ● fx-t fx-tok oken enmi mine nerr-v1 v1.3 .32 2 -C -P for solo mining ● crea create te an an ether ethereum eum priva private te key key ( https://www.myetherwallet.com/ https://www.myetherwallet.com/ ) ) ● send some some ether ether ( 10 USD ) to the corresponding corresponding public key ( you need gas to create blocks ) ● requ reques estt a free free api api key key on on https://infura.io/ ● edit edit tok token enmi mine ner. r.iini ○ Host=h Host=https ttps://m ://main ainnet. net.inf infura ura.io .io/ EY> ○ RPCPort=443 ○ Miner MinerAc Acct ct=0 =0x0 x0000 0000y 0you ourp rpubl ublic icke key y ○ Acct AcctPK PK=0 =0000 0000y 0you ourp rpri riva vate teke key y ● fx-t fx-tok oken enmi mine nerr-v1 v1.3 .32 2 -C -S ● send your your coins coins to another address address after mining mining ( as the private private key key lies lies on your your pc around )
this is the output when everything is ok:
when the mining crashes, two things happen ● the power power usa usage ge goes goes down down ( 200W 200W -> -> 20W 20W ) ● there there are no no more more “Hash” “Hash” outp outputs uts,, only only “Bloc “Block” k” output outputs: s:
when this happens and you restart the software, it does no longer work:
you then have to reprogram the bitstream. To do so you also have to increase voltage back to 0.72V. So you certainly want to prevent this - very annoying.
sometimes this happens: “Max Nonce: Failed to write new vector to fpga” ( ~48 hours for me)
create an endless loop batch file to restart after errors like above:
You should put a shortcut in windows autostart ( several batchfiles for multiple cards ) - so it will continue mining after reboot / windows update ( dont’t forget to choose windows as standard in multi-boot multi-boot systems systems )
clocks Bitstream
V
error rate
Power@Wall [W]
idle
0
programming
20
11.5 GH
0.72
0%
210
11.5 GH
0.69
0%
195
11.5 GH
0.66
0%
180
11.5 GH
0.64
0%
170
11.5 GH
0.63
0,2 %
160
11.5 GH
0.62
1%
155
11.5 GH
0.61
3%
150
13 GH
0.72
0%
240
13 GH
0.69
0%
220
13 GH
0.66
0%
205
13 GH
0.64
0.1 %
195
13 GH
0.63
0.5 %
185
13 GH
0.62
3%
180
13 GH
0.61
6%
175
<- Optimum?
<- Optimum?
there are 2 types of errors: ● board is is still still hashing, hashing, but but error error rate increases: increases: this this value value can can be tuned tuned here ● board board is cras crashed. hed. for me me it cras crashed hed much much less less on on 11.5 GH summary: Pro
Contra
0.85 V
No Dongle necessary. no crashes
very hot card
0.72 V
quite stable and cold
0.62 V
very cool
a word about infura.io :
much interaction needed, reprogramming
sometimes is see connection errors ( libcurl ), so we will sometimes loose blockchain data, maybe can not submit or get stuck at donation mining ( pool ) this is nothing unusual on a free cloud service. for best results ( especially with many cards ), you should set up a etherum full node! so you have always a reliable connection ( and may even reduce orphan blocks ) you have to install a ethereum node and open the rpc port to the whole internet ( be careful no password , so don't hold funds in that node ) here is my docker command line which would wo uld do this: docker run --restart=unless-stopped -d -v /path/to/your/data/storage:/root -p 8545:8545 -p 30303:30303 ethereum/client-go --rpc --rpcaddr 0.0.0.0 --rpccorsdomain "*" --syncmode full
Air cooling comparison: 10 GH, 0.73V ( ~ 180W ) °C
Miner
Temp1
Temp2
high Airflow
68
71
80
low Airflow
81
84
90
0xToken Altcoins ( 13.09.2018 ) S = 10 GH/s for simplicity Income = S * 3600 * 24 *30 / ( Diff * 2^22 Hashes ) * Reward * Price http://www.wolframalpha.com/input/?i=3600*24*30%2F+(+864881523+*+2%5E22+)+*+1000 0000000+*+50+*+0.00314+*+190 but we have to pay an ETH network fee for every created block: Tx fee = Blocks * 0,1 $ = S * 3600 * 24 *30 / ( Diff * 2^22 Hashes ) * 0,1 $ http://www.wolframalpha.com/input/?i=3600*24*30%2F+(+864881523+*+2%5E22+)+*+1000 0000000+*+0.1 1 ETH = 190 USD Diff Minimum = 100% Attack = 1 Block in 10 Minutes: S * 600s / 2^22 = 1430511 Coin
Diff
Reward
Price [ETH]
Exchange
$/Month
$ Fee
0xBTC
864881523
50
0.00314
Mercatox
213
-0.7
Atlantis
472078 1430511
500
0.0000000 01
enclaves.io
0.4
-432
0xDiary
1 1430511
4
0
-
0
-432
0xSkorch
63160 1430511
50
0
-
0
-432
Kiwi
18416 1430511
5000
0.0000000 155
enclaves.io
63
-432
0xLTC
28 1430511
50
0
-
0xZIBI
0
-
0xDOGEC OIN
0
0xRamen
0
0xbcash
0
None of these Altcoins does doe s make any sense. All devs left. Just all are 0xbtc copy scams. You could mine like 95% of each coin - but you would have to pay around 400 Dollar in Etherum Network fees. The following Table will proof that
Token
Tx of whole Contract
usage
0xBTC
185000
medium
Atlantis
31080
low
0xDiary
7533
low
0xSkorch
10457
low
0xLTC
38443
no, last 60 days without tx
Kiwi
114098
medium
0xZibi
2901
no
0xDOGE
14155
no, last 80 days without tx
Ramen
2112
no
0xbcash
65283
low
Mining Tribus This needs Linux. - instal installl Wind Windows ows 10 ( for dual boot, boot, half half disk disk size size ) - inst instal alll Lin Linux ux Mint Mint ( Res Restt of of the the Disk Disk ) - install updates - instal installl teamvie teamviewer wer .deb .deb ( start start with with system system,, lan only, only, define define passw password, ord, no no random random password) - inst instal alll viv vivad ado o lab lab edi edition tion - down downlo load ad ( fre free e acc accou ount nt need needed ed)) https://www.xilinx.com/member/forms/download/xef.html?filename=Xilinx_Viv ado_Lab_Lin_2018.2_0614_1954.tar.gz - sudo ./xsetup - install usb usb dr drivers - cd /opt/Xilinx/Vivado_Lab/2018.2/data/xicom/cable_drivers/lin64/install_script/ins tall_drivers - sudo sudo ./i ./insta nstalll_dr l_driv iver ers s - replug cables - set set usb usb perm permis issi sion ons s for for “use “user1 r123 23”” - sudo sudo gpa gpass sswd wd --a --add dd use user1 r123 23 dia dialo lout ut - start vivado - source source /opt/Xi /opt/Xilin linx/V x/Viva ivado_L do_Lab/2 ab/2018. 018.2/s 2/setti ettings6 ngs64.s 4.sh h - vivado_lab & - open open hard hardwa ware re mana manage gerr ( see see other other part part of man manual ual ) - progr program am bits bitstr trea eam m ( vcu152 vcu1525_t 5_tri ribus bus_3_ _3_650 650.b .bit it ) - rebo reboot ot syst system em ( impo import rtan antt ) - download mi miner - https://github.com/digitalcruncher/keccak-tribus - extract - cd vcu15 vcu152525-ke kecc ccak ak-t -tri ribu buss-va vari riab able le/c /ccm cmin iner er - sudo sudo apt apt-g -get et ins insta tallll libc libcur url3 l3 -y -y - edit tr tribus.sh - expo export rt LD_L LD_LIB IBRA RARY RY_P _PAT ATH= H=./ ./ - echo echo 1 > /sys/b /sys/bus/ us/pci pci/dev /device ices/0 s/0000: 000:03: 03:00.0 00.0/ena /enable ble - cp ./li ./lice cens nse.t e.tri ribus bus ./lic ./licens ense.t e.txt xt - ./activator - sleep 1 - ./ccmi ./ccminerner-kec keccak cak-tr -tribu ibus s -a tribus tribus -o stratum+ stratum+tcp tcp://y ://yiim iimp.e p.eu:85 u:8533 33 -u DF8V3HXmWgs9WrvJpCKVK18zm69HsAtLKP -p c=DNR - edit co cores.txt - 1 ( set set higher higher for more more speed speed -> -> needs needs more more cooling cooling and a licence licence ) - edit de devices.txt - /sys /sys/bu /bus/ s/pc pci/ i/dev devic ices es/00 /0000: 00:03 03:00 :00.0/ .0/re reso sour urce ce0 0 - run - sudo ./ ./tribus.sh
how to find the correct pci port: ( replace /sys/bus/pci/devices/0000:03:00.0/resource0 ) lspci | grep Xilinx 02:00.0 Serial controller: Xilinx Corporation Device 6a90 02:00.1 Serial controller: Xilinx Corporation Device 6a8f 03:00.0 Serial controller: Xilinx Corporation Device 6a90 03:00.1 Serial controller: Xilinx Corporation Device 6a8f -> try these four numbers until it works: 02:00.0, 02:00.1 , 03:00.0, 03:00.1
how to get licence.tribus? -> contact DigitalCruncher. You will need it if you want to run more than 1 core.
errors? ./ccminer-keccak-tribus: /usr/lib/x86_64-linux-gnu/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by ./ccminer-keccak-tribus) -> your linux uses libcurl4, whereas the binary needs libcurl3 -> sudo add-apt-repository http://security.ubuntu.com/ubuntu xenial-security main sudo apt-get install libcurl3
crash -> do not run vivado and ccminer at the same time!
wokring miner:
not working miner:
here we see a problem with the activator: for a valid activation, the “Status” after the line “unlock” should beginn with “0xC00” and not “0x800” . so if you have a valid activation code, try the following ● remo remove ve act activ ivato atorr from from your your sh-f sh-filile e ● run acti activat vator.s or.sh h manual manually ly ( edit edit for for your your pcie pcie slot slot ) ● run it seve several ral times times unti untill you you have have “0xC “0xC00” 00” ● dont dont run run it any any more more after afterwa ward rds s
Tribus power usage ( power hard to mease, jumps much ) cores
GH
V
Power @Wall
1
0,6
0.85
105 W
1
0,6
0.72
75 W
1
0,6
0.62
no shares
2
1,2
0.75
160 W
2
1,2
0.72
150 W
2
1,2
0.70
140 W
less shares
2
1,2
0.65
no shares
no shares
3
1,8
0.70
200 W
<- optimum
Tribus Altcoin Overview - Value of one Block ● 2 Denar Denariu ius s = 0,70 0,70 € ( sev sever eral al exc exchan hanges ges ) ○ after after 2500 2500 Bloc Blocks ks you can create create a mast masterno ernode de ● 1 BZL BZL = 0,0 0,01 1 € ( Cry Crypto ptobr brid idge ge ) ○ after after 2000 2000 Bloc Blocks ks you can create create a mast masterno ernode de ● 22.4 22.4 Scri Scriv v = 0,0 0,02 2 € ( sha shady dy exc excha hange nge ) ○ after after 4500 4500 Bloc Blocks ks you can create create a mast masterno ernode de not recommended: ● 2 Zula Zula = 0,001 0,001 € ( Crypt Cryptobri obridge, dge, but dev dev sile silent nt = dead dead ) ● 6 Virtus Virtus = 0,001 0,001 € ( Cryp Cryptobr tobridge idge,, but dev dev silent silent = dead dead ) ● 5 Tinco Tincoin in ( not listed listed,, dev dev silen silentt = dead ) ● 300 Vegac Vegacoi oin n ( long long time time dead dead ) ● 0.8 0.8 Key Keyc co ( POW POW ende ended d) ● 0.05 Aver Averopay opay ( not liste listed. d. it has has 99,5% 99,5% premin premine e - absolut absolute e scam scam ) ○ after after 100000 100000 Bloc Blocks ks you you can can create create a mast mastern ernode ode (wtf. (wtf... ) https://crypt0.zone/coins/tribus conclusion: Only Denarius, BZL and Scriv are useful profit-switching coins. (exchangeable). Only Denarius has long term potential.
You could mine Zula or Virtus and hope for a pump ( create now sellorders at Cryptobridge with 10x of the actual price. check in 6 months )
important ccminer command line options
(normally visible via -h ) -f, --diff-factor Divide difficulty by this factor (default 1.0) -m, --diff-multiplier Multiply difficulty by this value (default 1.0) -o, --url=URL URL of mining server -O, --userpass=U:P username:password pair for mining server -u, --user=USERNAME username for mining server -p, --pass=PASSWORD password for mining server --cert=FILE certificate for mining server using SSL -x, --proxy=[PROTOCOL://]HOST[:PORT] --proxy=[PROTOCOL://]HOST[:PORT] connect through a proxy -t, --threads=N number of miner threads (default: number of nVidia GPUs in your system) -r, --retries=N number of times to retry if a network call fails (default: retry indefinitely) -R, --retry-pause=N time to pause between retries, retries, in seconds (default: 15) --shares-limit maximum shares to mine before exiting the program. --time-limit maximum time [s] to mine before exiting the program. -T, --timeout=N network timeout, in seconds (default: 300) -s, --scantime=N upper bound on time spent scanning current work when long polling is unavailable, in seconds (default: 5) --submit-stale ignore stale job checks, may create more rejected shares --no-gbt disable getblocktemplate support (height check in solo) --no-longpoll disable X-Long-Polling support --no-stratum disable X-Stratum support -q, --quiet disable per-thread hashmeter output --no-color disable colored output -D, --debug enable debug output -P, --protocol-dump verbose dump of protocol-level activities -b, --api-bind=port --api-bind=port IP:port for the miner API (default: (default: 127.0.0.1:4068), 0 disabled --api-remote Allow remote control, like pool switching, imply --api-allow=0/0 --api-allow=... IP/mask of the allowed api client(s), 0/0 for all --max-temp=N Only mine if gpu temp is less than specified value --max-rate=N[KMG] Only mine if net hashrate is less than specified value --max-diff=N Only mine if net difficulty is less than specified value --keep-clocks prevent reset clocks and/or power limit on exit --hide-diff Hide submitted shares diff and net difficulty -B, --background run the miner in the background --benchmark run in offline benchmark mode -c, --config=FILE load a JSON-format configuration file can be from an url with the http:// prefix -> maybe it is possible to define a second (backup) pool in config file? -> otherwise, create a shell script where you start one ccminer after the other, with several pools, with “-r 3”, as endless loop. ( so when a pool is down, it will be tried 3 times, then ccminer closes )
ccminer api access
it is nice to monitor your farm in a central tool like awesome miner. so you can define in ccminer “--api-bind 0.0.0.0:1080” to listen on port 1080. awesome miner could also connect and shows hasrate. but then i had a bug: ccminer crashed with the comment: [2018-08-18 08:57:58] Your system does not support CUDA 8.0 API! so i guess, api is just not fully implemented in this software
Mining Keccak https://crypt0.zone/coins/keccakc Altcoins: Worth of one Block ● Keccak ○ 59 Smart martCa Cash sh = 2,8 2,80 0€ ○ 8 Ma Maxcoin oin = 0,2 0,28 € ○ Slothc Slothcoin oin (SLOTH (SLOTH)) – Inactiv Inactive e ( but pool: pool: https://pool.atomminer.com/ https://pool.atomminer.com/ ) ) ○ 365C 365Coi oin n (36 (365) 5) – Ina Inact ctiv ive e ○ Gall Galleo eon n (GL (GLN) N) – Ina Inact ctiv ive e ○ Heli Helix x Coin Coin (HX (HXC) C) – Inac Inacti tive ve ○ Cryp Crypto toMe Meth th (MET (METH) H) – Inac Inacti tive ve ○ Bitcoi Bitcointa ntalkC lkCoin oin (TALK) (TALK) – Inacti Inactive ve ( but Pool Pool:: https://pool.atomminer.com/ https://pool.atomminer.com/ ) ) ○ XDNA? ● Keccak-C ○ 34 Crea Creati tive veco coin in = 1,35 1,35 €
Mining Zenprotocol != Zencash ( aka Horizen, equihash ) https://www.zenprotocol.com/en https://coinmarketcap.com/currencies/zen-protocol/ new miner from 21.09.2018: download binary and bitstream here: https://github.com/digitalcruncher/zen-protocol
Altcoins: Worth of one Block ● Keccak-ZP ○ 50 ZP = 25 $ Explorer: https://zp.io/ Pools: https://zp.leafpool.com/ -> https://zp.leafpool.com/ -> has 95% of Network. 2% fee https://zp.pandapool.pw/ -> https://zp.pandapool.pw/ -> down https://pool.atomminer.com/ -> https://pool.atomminer.com/ -> “red” ? Profit Estimation: Network: 8000 GH 1 VCU: 12 GH 100 Blocks = 6,5 h 24h = 400 Blocks 1 VCU = 400/8000*12 Blocks = 0,6 Blocks VCU: 15$ / day Leafpool: Stratum Hostname/Location: US: zp-us-east.leafpool.com EU: zp-eu.leafpool.com Asia: zp-asia.leafpool.com Stratum Ports: GPU Mining: 8811 Rigs/Mining Farm: 8822
NVIDIA (CUDA) Miners ccminer-zp Github Download Link
ccminer -o stratum+tcp://zp-us-east.leafpool.com:8811 -u YOUR_ADDRESS.WORKER_NAME -p x
./ccminer-zp -o stratum+tcp://zp-eu.leafpool.com:8822 -u zen1q9xu2lxr55hucs36vc5vmydqj04y9vdgp3zx5n94l0mtwn9a2cs4sarn3t2.yourDonationNa me -p x -q
Mining Zenprotocol Cores
Speed
Voltage
Power
T ( Immersion )
21
12 GH
0.73 V
280W
58 / 68 / 73 °C
Solo Mining Why? Leafpool is doing 100% attack on network. No Fee. ● ●
install no node: https://docs.zenprotocol.com/preparation point point miner miner at own node: node: 127. 127.0.0. 0.0.1:11 1:11567 567 (? testing testing..) ..)
Profit Estimation last update: 11.08.2018
Coin
Bitstream
Hashrate
Voltage
power[W]
blocks/day
$ / month
0xBTC
Zetheron
13 GH
0,62
160
0,25
200
Denarius
DigitalCruncher
1,95 GH
0.75
240
8
180
Maxcoin
DigitalCruncher
12,6 GH
?
?
9
100
https://0x1d00ffff.github.io/0xBTC-Stats/?page=miningcalculator&# https://whattomine.com/coins/187-dnr-tribus?utf8=%E2%9C%93&hr=1950.0&p=200&fee=0. 0&cost=0.3&hcost=0.0&commit=Calculate https://whattomine.com/coins/73-max-keccak?utf8=%E2%9C%93&hr=12600&p=200&fee=0. 0&cost=0.3&hcost=0.0&commit=Calculate
Real World income 28.08.2018 - here are the results of running one cards for one week. i can confirm that this configuration runs really stable ( no crash in that week )
Coin
Bitstream
Hashrate
[V]
power[W]
$ / month
0xBTC
Zetheron
10 GH
0.75
160
140
DNR/SCRIV/BZL Switching
DigitalCruncher 1,3 GH 2 Cores
0.75
160
240
Next Steps ● ● ● ●
look at at immers immersion ion cooli cooling: ng: oilba oilbath th = best best possibl possible e cooling cooling look look at new new bit bitst stre ream ams s get mo more ca cards develop develop solo solo mini mining ng pool pool ( no averag averaging ing over over profi profit, t, less less fee fee ) ○ develo develop p profit profit switc switchin hing g ( mine mine one algo algo but diff differen erentt coins coins ) ○ develo develop p algo swit switchi ching ng ( comman command d line line reporgr reporgramm amming? ing? )
Immersion Cooling Theory you need a dielectric liquid ( similar to oil ). you put the hardware in it. the liquid can take much more heat than air. you then bring the hot liquid away, cool it, and bring it back ● 2 phas phase e cool coolin ing: g: boi boililing ng liq liqui uid. d. ○ if boiling boiling point is e.g. 61°C 61°C and and you put the the board inside inside the liquid, liquid, the liquid liquid ( and the card ) can not get hotter than 61°C ( thermodynamics ) ○ you have have to put put it inside inside a closed closed chamber, chamber, with with cooling cooling pipes ( cold water water flowing through ) ○ the boiled boiled liquid liquid will will rise rise to the top of the chamber and meet meet the pipes: pipes: when when they are colder than 61°C, the gas will become liquid again and fall down to the ground. ○ compli complicat cated ed to realiz realize. e. the boili boiling ng might might be bad for for the hardwar hardware e ○ http://multimedia.3m.com/mws/media/644792O/thermal-management-fluids-fo r-the-electronic-industry.pdf ● 1 phase phase coolin cooling: g: the the liqu liquid id just just gets gets warm. warm. ○ you you can can use use an an ope open n bat bath h ○ but you have to thank thank about about how how to cool cool the the liqui liquid d 1 phase will be good for mining / fpgas. so we use this.
Coolant which material should be used? ● transfo transforma rmator tor oil oil is is well well known known for electr electronic onics s ○ but itit might might damage damage the tiny computer computer component components s ( corrosion, corrosion, chemical chemical reaction ) ● there there are some some mater material ials s which which could could fit fit fpga fpga board boards s ○ http://multimedia.3m.com/mws/media/64888O/fluorinert-electronic-liquid-fc-40 .pdf ○ https://www.engineeredfluids.com/electrocool
FC-40
EC-100
boiling point [°C]
165
flash point [°C]
294
180
vapor pressure [Pa]
287
“low”
density [kg/m³]
1.8
0.81
BC-888
better = more
130
more less
0.82
not 1(water)
viscosity [cSt]
2.2
9.58
10
less (for pump)
heat [J/(K kg)]
1100
2203
2203
more
conductivity[W/(m C)
0.065
0.13
0.13
more
price / liter
500 $
17 $
9$
The company “Engineered Fluids” has recommended me to use BC-888 also for fpga. this and the cheap price told me to buy it. shipping to europe is quite expensive ( 190USD / 20l ), but i had no other choice.
Heat Removal There are several devices which bring the heat from a liquid to the air. here a comparison: “Water Chiller” ● Heat Pump: Pump: You You take take e.g. 400W electri electricit city y to take take 1200W 1200W away from the water ( and bring 1600W to the air ) ● with with fancy fancy contr controll oller er to keep keep tempe temperat rature ure at at 27.34 27.34 °C ifif you want ● you brin bring g the cool coolant ant to to a tempera temperatur ture e even even lower lower than than air air temperature ● overkil overkill: l: we don’t don’t need need such such cold cold liquid liquid ● cons consum umes es too too muc much h ene energ rgy y ● 100 1000 € for 100 1000W “Radiator” ● car car comp compon onent ent (ch (chea eap), p), for for wate waterr or oil oil ● no rea reall doc docum ument entat atio ion n avai availa labl ble e ● this this pass passiv ive e cool coolin ing g might might be enou enough gh ● 100 €
“Air Heater” ● for for hea heati ting ng the the air air in a hous house e ● its its a coppe copperr pip pipe e conn connec ecte ted d to to aluminium fins ● their their capa capaci city ty sta start rts s from from 10k 10kW W ( we we don’t need that much - but this tells us that every Watt we bring in, will go to the air ) ● they they have have a inbu inbuililtt fan fan ● we bri bring ng the the coo coola lant nt tem temper perat atur ure e just just above air temperature ● they they hav have e a datas datashe heet et - 500 500 € for for “10kW”
“Heat ● ● ● ● ●
Register”(german - “damper register” ? ) simi simila larr to to “Ai “Airr hea heate ter” r” same same cool cool data datash shee eets ts it’s it’s just just the the pip pipe e wit withou houtt a fan fan you you con conne nect ct your your own own fan fan pro: you can direct direct the hot air ( use use a hose to bring it out the house ) ● 300 300 € for “10 kW”
data sheets: the concrete cooling depends on several factors. the data sheets explain the relation between: ● air flow V ● heat power PT ● air air inta intake ke tem tempe pera ratu ture re Tp1 Tp1 ● air air exi exitt tem tempe pera ratu ture re Tp2 Tp2 ● liqui liquid d inta intake ke temper temperatu ature re Tw1 Tw1 ● liqui liquid d exit exit temp temper erat atur ure e Tw2 Tw2 ● liquid flflow Qw Qw
●
drop in press pressure ure dpw dpw ( your your pump pump need needs s to be comp compens ensate ate this this )
how to read it? If ● the air air flo flow w rat rate e is is V = 210 2100 0 m³/ m³/h h ● the the fpga fpga con consu sume mes s PT= PT= 1,9k 1,9kW W ● the the liq liqui uid d flow flow is is QW QW = 82l 82l/h /h ● the room room tempe tempera ratur ture e is is Tp1 Tp1 = 20°C 20°C then ● the liquid liquids s hottes hottestt tempe temperat rature ure is Tw1 Tw1 = 60°C ● the liquid liquids s lowes lowestt tempe temperat rature ure is Tw2 Tw2 = 40°C ● the exit exit air air wil willl be be Tp2 Tp2 = 22, 22,5°C 5°C if room temperature is 15° C AND liquid flow increases to 135 l, then you can consume 3,1kW, and have still Tw1 = 60°C so it is not so easy to read: but i would say: if you keep the 1,9kW, fixed room temperature and increase flow rate ( my pump: 1800 l/h ), things will get much colder.. so you can even decrease airflow V and make it silent.
My conclusion on heat removal:
Chiller is waste of energy. radiator too speculative. My first intent was to bring an air heater outside the building, circulate liquid to fpgas inside the building. But this has a strong disadvantage in the winter: imagine 25°C room temperature, -5°C outside temperature. you can now easiliy chill the liquid to Tw1 = 0°C, Tw2=5°C. so your open box of liquid is 5°C cold. your liquid hose is also not much warmer on the outside. Nice. But our problem is air moisture: liquid water will form on the surfaces and drip into your BC-888. This pollution might be bad for the cards.
So i decided to use a simple pipe fan and a heat register: they are inside the building. so the lowest Temperature Tw2 can be the room temperature Tp1 ( thermodynamics ). Tw1 will always be higher as Tw2, so no problems with moisture. The hot air will then be brought outside the building:
Fan My Fan has V = 1040 m³/h, 345 Pa pressure, 52 dB loud, 108W, 95€ https://www.iventilatoren.de/rohrventilator-dalap-ap-profi-200-mit-schalter-id1105 you always have to look at the pressure curve:
how to to read? you first have to check the air drop of your system: the heater fins have a air resitance ( sometimes also in the data sheet ): if it is 100 Pascal, and you use “min”, then airflow will be 650m³/h ( not the advertised value of 830 ! ) air resistance also depends on flowrate, so a real world system will be the crossing of the system curve with the ventilation curve:
Pump You have to circulate the liquid from the fpga to the heat register. i chose a pump for brewing beer ( 95°C allowed, FKM seals ) https://www.polsinelli.it/en/electric-pump-for-beer-novax-20-b-P1041.htm a pump has a similar curve for pressure drop over flow rate. so our pump with 1500 l/h will again be slower ( 1000l/h ? )
How many circuits? There are several options: ● you circ circula ulate te BC888. BC888. you you need need one pump pump and and one heat heat regist register. er. ○ very eff effective ○ need eed more BC888 ● you circula circulate te BC888 BC888 with an heat exchanger. exchanger. you you have second ( water ) circle, circle, which which circulates from the heat exchanger to the heat register ○ high higher er temp temper erat atur ure e for for BC88 BC888 8 ○ you can can choose choose stand standard ard / used used compo component nents s for the the water water circl circle e ● …
my immersion cooling setup
second setup:
● ● ●
●
BitCo tCool BC-88 C-888 8 VCU1525 HDPE HDPE tank tank 31l ( for brewin brewing g beer beer ) - will will fit fit like like 4 FPGAs FPGAs https://www.bauhaus.info/sonstige-gartenhelfer/floraworld-classic-maischefass/p/161 25460 mortar mortar buck buckets ets as as backup backup tank tank (not (not safe safe for hot hot tempera temperatur tures) es) https://www.bauhaus.info/moertelkuebel-baueimer/probau-moertelkuebel/p/1394034 9 ○ cheap cheap but they they smel smelll lik like e hel helll
●
●
●
● ● ● ● ●
●
● ●
●
1500 lph, lph, 270W 270W Pump Pump ( for for brewin brewing g beer ) with with FKM FKM seal seals s for up to to 95°C https://www.polsinelli.it/en/electric-pump-for-beer-novax-20-b-P1041.htm ○ italia italian n 20mm 20mm fluid fluid conne connecto ctorr - will will fit fit a ¾ inch inch hose hose ○ high high vol volum ume, e, but but ver very y lou loud d Standar Standard d PC with with 8 GB RAM, RAM, SSD, energ energy-e y-effi fficie cient nt mid-r mid-range ange CPU CPU https://www.csl-computer.com/shop/product_info.php?products_id=15016&cPath=11 _353 850 850 W Plat Platiinum num PSU PSU https://www.caseking.de/kolink-continuum-80-plus-platinum-netzteil-modular-850-wat t-mit-kaltgeraetekabel-gebu-201.html PCIe PCIe riser isers s + cabl cables es 8 kg kg hea heatt re register https://www.abluft24.de/heizregister/warmwasser-heizregister-fuer-rundrohrsysteme high high volume ume fan fan https://www.iventilatoren.de/rohrventilator-dalap-ap-profi-200-mit-schalter-id1105 exha exhau ust ai air ho hose fuel hose hose 19mm 19mm 27€/m 27€/m ( safe safe choic choice e for for BC-8 BC-888 88 ) https://www.landefeld.de/gruppe/en/tank-truck-suction-pressure-hose-nbr-en-12115/T WSSP19 water hose hose 19mm 19mm 4€/m ( cheap choice for BC-888 BC-888 - PVC might might be a problem? problem? ) https://www.landefeld.de/gruppe/de/pvc-schlaeuche-mit-gewebeeinlage-lebensmittel qualitaet/TX4 adaptor adaptors s for connec connectin ting g 19mm hose hose to 3/4 3/4 ‘’ regist register, er, brack brackets ets vcu vcu bac backsi kside coo cooli ling ng ○ https://www.reichelt.de/waermeleitfolie-doppelseitig-klebend-100x100mm-wl-f olie-404-11-p112269.html?r=1 ○ good fan: https://www.reichelt.de/luefter-40x40x20mm-12-v-0-115-a-fan-ml-4020-12-p7 6106.html?r=1 ○ cheap fan: https://www.reichelt.de/luefter-12v-dc-40x40x10mm-u-min-5800-fan-4010-12v -p11004.html?r=1 ○ https://www.reichelt.de/profilkuehlkoerper-100x100x40mm-1-3k-w-v-pr173-10 0-p35401.html?r=1 steel container container ( for for food , “Gastronor “Gastronorm m 1/1”), 1/1”), of course course the version version without without holes: 53 x 32.5 x 20 cm https://www.metro.de/cat/de/products/product/440229001001/APS-GN-Beh%C3%A4l ter-1/1-200-mm-Edelstahl-perforiert
video cooming soon ( first week of september )
first important results: ● its fun fun to buil build d this. this. ( but but the the BC-888 BC-888 no no fun - very very oily oily ) ● you get get very nerv nervous ous when when someth something ing crash crashes. es. but but then you you are happy happy when when everything runs ● you do need need the metal cooling cooling blocks blocks intended intended for for air - the pure pure chip chip will will go to 100°C in 5 seconds, even in the fluid
● ●
●
●
●
i would would leave leave the stock stock versio version n of the passive passive card card just as it is - just remove remove the backside cover cooling cooling the liquid liquid ( heat register register ) is absolutel absolutely y not crucial crucial at the the moment moment - with one one card in 20 liters you just make it a little bit warmer than ambient temperature - so it might even be okay to leave that part away and just use heat dissipation ( warm fluid gives heat directly to air over its big surface ) ○ the backside backside of the card ( read with with LT box box ) is is directly directly cooler than in air the liquid liquid stays stays very very cold: cold: like like 4°C 4°C above room temperat temperature ure ( when you you measure measure anywhere in the tank, not directly near the card ) ○ you bett better er choose choose a silent silent pump pump:: less less flow flow will will be ok ○ smal smalle ler, r, sile silent nt fan will will be ok it is very crucial crucial to have have a good “fluid flow” - ( like like good air air flow flow for fans ). ). so you you take the output hose of the tank and place it directly on the heat sink of the fpga - then the chip always sees fresh, cool fluid - it has so possibility of collecting heat and getting warm ○ bad flui fluid d flo flow: w: 80°C 80°C in in viv vivado ado ○ good good flu fluid id flow flow:: 50° 50°C C in in viv vivado ado small small fans fans:: bigger bigger fins fins is is better better.. maglev maglev is is better better
0xBTC - 0.74V - 10 GH comparison:
Miner
LT Temp_1
LT Temp_2
Airflow bazooka
70°C
76 °C
80 °C
airflow antminer
70
73
76
Fluid without flow
83°C
55°C
75°C
Fluid with flow
48°C
52°C
64°C
0xBTC - overclocking
GH
Volt
Watt
Error
Miner
Temp_1
Temp_2
10
0,74
190
0%
48°C
52°C
64°C
10
0,72
170
0%
46°C
52°C
62°C
11,5
0,72
210
0%
51°C
52°C
67°C
13
0,72
235
0%
52°C
54°C
71°C
15
0,72
crash
15
0,75
340
15
0.78
380
15
0,8
390
17
0,8
380
0%
57 °C
57 °C
82 °C
60
60
85
33 %
60 °C
60 °C
88 °C
33 %
58 °C
60 °C
90 °C
With ~ 370 Watt Power consumption we see in LT FAULT_WARN_LIST = “IOUT_OC_FAULT”, STATUS_IOT = (0x80) IOutOverCurrentFault . this makes totally sense.. this is just too much current. remember that we have one 8-pin and one 6-pin power connector. so PCIe power specifications allow us to draw 150W + 75W = 225W ! at the moment this is too much for me.. also Temp_2 gets too high - this is probably because i flow cool only the front side of the fpga - not the back side.
steel container 0xbtc 0.73V
GH
Miner
LT Temp_1
LT Temp_2
10
50°C
58 °C
60 °C
11,5
54°C
62°C
66°C
13
55°C
67°C
70°C
15 ( 0.74V )
70
85
90
15 ( hotter card )
80
90
95
tribus - overclocking
cores
MH
Volt
Watt
Miner
Temp_1
Temp_2
1
650
0,75
70
43°C
50°C
55°C
2
1300
0,75
150
52°C
54°C
63°C
3
1950
0,75
245
58°C
56°C
76°C
3
1950
0,73
230
55°C
54°C
73°C
4
“2600”
0,73
230
55°C
54°C
73°C
as we see, there is a maximum of 3 useable cores.
Nexus Algo ( 18.10.2018 )
MH
Volt
Watt
Miner
Temp_1
Temp_2
1500
0,86
350
75°C
86°C
94°C
400
0,60
Real World income 02.09.2018 Coin
Bitstream
Hashrate
[V]
power[W]
$ / month
0xBTC
Zetheron
13 GH
0.72
235
200
DNR/SCRIV/BZL Switching
DigitalCruncher 1,95 GH 3 Cores
0.73
230
450
Coin
Bitstream
Hashrate
[V]
power[W]
$ / month
0xBTC
Zetheron
13 GH
0.72
235
200
DNR/SCRIV/BZL Switching
DigitalCruncher 1,95 GH 3 Cores
0.73
230
280
Coin
Bitstream
Hashrate
[V]
power[W]
$ / month
0xBTC
Zetheron
13 GH
0.72
235
300
0xBTC
Zetheron
15 GH
0.74
360
350
DNR/SCRIV/BZL Switching
DigitalCruncher 1,95 GH 3 Cores
0.73
230
320
11.09.18
22.09.18
ZP
DigitalCruncher 12 GH 21 Cores
0.73
280
350
next step i will create smaller tanks - one tank per fpga. it will be a lot eaiser to handle. you can control the flow of the liquid better. each gets its own small ( quiet ) pump with 13mm flexible hose. i will pull the fluid from the back of the card ( create a little liquid flow there ) and pump it in the inside of the card ( big liquid flow here ). the fluid will go through an heat exchanger. then we have a second circle of fluid - this can just be water with standard components ( pump, heat exchanger ). the circle will go through all the single heat exchangers of the BC888-circles.
